[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:22 dist-test-slave-lgr5 krb5kdc[22239](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:22 dist-test-slave-lgr5 krb5kdc[22239](info): set up 2 sockets
Aug 12 07:09:22 dist-test-slave-lgr5 krb5kdc[22239](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260812 07:09:24.127404 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.002s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:24 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:24.291167 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37595
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:37595
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:24.623587 22259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:24.623872 22259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:24.624015 22259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:24.631407 22259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:24.631528 22259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:24.631616 22259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:24.631703 22259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:24.631773 22259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:24.641494 22259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:37595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:24.642753 22259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:24.644074 22259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:24.658432 22266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:24.658475 22269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:24.658499 22267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:24.659269 22259 server_base.cc:1034] running on GCE node
I20260812 07:09:24.660111 22259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:24.661625 22259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:24.662843 22259 hybrid_clock.cc:648] HybridClock initialized: now 1786518564662810 us; error 61 us; skew 500 ppm
I20260812 07:09:24.663287 22259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:24 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518564, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:24.666450 22259 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:24.668210 22259 webserver.cc:460] Webserver started at http://127.21.181.126:44331/ using document root <none> and password file <none>
I20260812 07:09:24.668892 22259 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:24.669004 22259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:24.669266 22259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:24.672518 22259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "d9063b2535fe47159b7757eefe3f9a6c"
format_stamp: "Formatted at 2026-08-12 07:09:24 on dist-test-slave-lgr5"
server_key: "113c0ef86acc90bbeade28b3c5fe4344"
server_key_iv: "f2cb47a8c530ad38c68152dc8e790cfb"
server_key_version: "encryptionkey@0"
I20260812 07:09:24.673439 22259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "d9063b2535fe47159b7757eefe3f9a6c"
format_stamp: "Formatted at 2026-08-12 07:09:24 on dist-test-slave-lgr5"
server_key: "113c0ef86acc90bbeade28b3c5fe4344"
server_key_iv: "f2cb47a8c530ad38c68152dc8e790cfb"
server_key_version: "encryptionkey@0"
I20260812 07:09:24.678879 22259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260812 07:09:24.682466 22281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:24.683779 22259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 07:09:24.683943 22259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "d9063b2535fe47159b7757eefe3f9a6c"
format_stamp: "Formatted at 2026-08-12 07:09:24 on dist-test-slave-lgr5"
server_key: "113c0ef86acc90bbeade28b3c5fe4344"
server_key_iv: "f2cb47a8c530ad38c68152dc8e790cfb"
server_key_version: "encryptionkey@0"
I20260812 07:09:24.684162 22259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:24.713203 22259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:24.717350 22259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:24.717624 22259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:24.737859 22259 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:37595
I20260812 07:09:24.737861 22387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:37595 every 8 connection(s)
I20260812 07:09:24.739323 22259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:24.742980 22388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:24.744973 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22259
I20260812 07:09:24.745270 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:24.747149 22229 external_mini_cluster.cc:1428] Setting key 3b1624d240e6ba91c0f40299efd4696e
I20260812 07:09:24.753588 22388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c: Bootstrap starting.
I20260812 07:09:24.756474 22388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:24.757552 22388 log.cc:824] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:24.760221 22388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c: No bootstrap required, opened a new log
I20260812 07:09:24.769873 22388 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9063b2535fe47159b7757eefe3f9a6c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37595 } }
I20260812 07:09:24.770205 22388 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:24.770326 22388 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9063b2535fe47159b7757eefe3f9a6c, State: Initialized, Role: FOLLOWER
I20260812 07:09:24.771072 22388 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9063b2535fe47159b7757eefe3f9a6c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37595 } }
I20260812 07:09:24.771371 22388 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:24.771517 22388 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:24.771690 22388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:24.773859 22388 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9063b2535fe47159b7757eefe3f9a6c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37595 } }
I20260812 07:09:24.774304 22388 leader_election.cc:302] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9063b2535fe47159b7757eefe3f9a6c; no voters:
I20260812 07:09:24.774868 22388 leader_election.cc:288] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:24.775065 22397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:24.776262 22397 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [term 1 LEADER]: Becoming Leader. State: Replica: d9063b2535fe47159b7757eefe3f9a6c, State: Running, Role: LEADER
Aug 12 07:09:24 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:24.776937 22397 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9063b2535fe47159b7757eefe3f9a6c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37595 } }
I20260812 07:09:24.777278 22388 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:24.780557 22399 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9063b2535fe47159b7757eefe3f9a6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9063b2535fe47159b7757eefe3f9a6c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37595 } } }
I20260812 07:09:24.781059 22399 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:24.781697 22400 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9063b2535fe47159b7757eefe3f9a6c. Latest consensus state: current_term: 1 leader_uuid: "d9063b2535fe47159b7757eefe3f9a6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9063b2535fe47159b7757eefe3f9a6c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37595 } } }
I20260812 07:09:24.782035 22400 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:24.785979 22409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:24.789722 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:24.751122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.21.181.126:37595)
0812 07:09:24.752103 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:24.752119 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:24.753565 (+ 1446us) server_negotiation.cc:407] Connection header received
0812 07:09:24.762651 (+ 9086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:24.762676 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:24.763121 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:24.763539 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:24.766424 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:24.768537 (+ 2113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:24.770236 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:24.771242 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:24.771427 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:24.778399 (+ 6972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:24.778410 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:24.778429 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:24.778484 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:24.783181 (+ 4697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:24.784385 (+ 1204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:24.784394 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:24.784402 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:24.784639 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:24.785255 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:24.785260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:24.785263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:24.786343 (+ 1080us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:24.786668 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:24.787749 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0812 07:09:24.788318 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":412,"threads_started":1}
I20260812 07:09:24.795585 22409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:24.803882 22409 catalog_manager.cc:1347] Generated new cluster ID: 175a23b7c8ec41479dd32eb8605751b5
I20260812 07:09:24.803992 22409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:24.811335 22409 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:24.812994 22409 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:24.824827 22409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9063b2535fe47159b7757eefe3f9a6c: Generated new TSK 0
I20260812 07:09:24.825721 22409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:24.871292 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37595
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:25.199568 22451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:25.199903 22451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:25.200112 22451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:25.208509 22451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:25.208622 22451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:25.208776 22451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:25.218433 22451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:25.219609 22451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:25.220928 22451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:25.233526 22456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:25.233520 22457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:25.233527 22462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:25.234267 22451 server_base.cc:1034] running on GCE node
I20260812 07:09:25.234903 22451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:25.235772 22451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:25.237015 22451 hybrid_clock.cc:648] HybridClock initialized: now 1786518565236935 us; error 57 us; skew 500 ppm
I20260812 07:09:25.237325 22451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:25 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:25.240466 22451 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:25.241873 22451 webserver.cc:460] Webserver started at http://127.21.181.65:44351/ using document root <none> and password file <none>
I20260812 07:09:25.242504 22451 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:25.242617 22451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:25.242900 22451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:25.245788 22451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "dab82c8f71884b1ab12684ac38879928"
format_stamp: "Formatted at 2026-08-12 07:09:25 on dist-test-slave-lgr5"
server_key: "91ba9d6483ef9006d2c07e2c9e9e60bf"
server_key_iv: "9b1e6bb402b2354320cec18f607d329b"
server_key_version: "encryptionkey@0"
I20260812 07:09:25.246505 22451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "dab82c8f71884b1ab12684ac38879928"
format_stamp: "Formatted at 2026-08-12 07:09:25 on dist-test-slave-lgr5"
server_key: "91ba9d6483ef9006d2c07e2c9e9e60bf"
server_key_iv: "9b1e6bb402b2354320cec18f607d329b"
server_key_version: "encryptionkey@0"
I20260812 07:09:25.251546 22451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:09:25.254818 22469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:25.256031 22451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:09:25.256218 22451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "dab82c8f71884b1ab12684ac38879928"
format_stamp: "Formatted at 2026-08-12 07:09:25 on dist-test-slave-lgr5"
server_key: "91ba9d6483ef9006d2c07e2c9e9e60bf"
server_key_iv: "9b1e6bb402b2354320cec18f607d329b"
server_key_version: "encryptionkey@0"
I20260812 07:09:25.256385 22451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:25.274446 22451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:25.278168 22451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:25.278478 22451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:25.279415 22451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:25.280786 22451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:25.280860 22451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:25.280980 22451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:25.281032 22451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:25.314659 22451 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:43639
I20260812 07:09:25.314702 22684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:43639 every 8 connection(s)
I20260812 07:09:25.315887 22451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:25.321173 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22451
I20260812 07:09:25.321401 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:25.322237 22229 external_mini_cluster.cc:1428] Setting key bb90b74ea9c5ba2cf8ea5406b4b44a95
Aug 12 07:09:25 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:25.335819 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:25.318293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:33241 (local address 127.21.181.126:37595)
0812 07:09:25.318734 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:25.318738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:25.319564 (+ 826us) server_negotiation.cc:407] Connection header received
0812 07:09:25.321643 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:25.321648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:25.321720 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:25.321843 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:25.323663 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.324675 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.326192 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.326744 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.326873 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:25.330377 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:25.330383 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:25.330387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:25.330422 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:25.333011 (+ 2589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:25.333803 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:25.333806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:25.333809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:25.333914 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:25.334323 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:25.334326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:25.334328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:25.334581 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:25.334692 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:25.335406 (+ 714us) server_negotiation.cc:299] Negotiation successful
0812 07:09:25.335569 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":231,"threads_started":1}
I20260812 07:09:25.337101 22690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:25.318730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37595 (local address 127.21.181.65:33241)
0812 07:09:25.319407 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:25.319443 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:25.321347 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:25.322145 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:25.322156 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:25.322859 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:25.323508 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:25.323534 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.324839 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.324843 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:25.326046 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:25.326059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.326926 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.326931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:25.327146 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:25.327859 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:25.327887 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:25.330158 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:25.333146 (+ 2988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:25.333153 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:25.333166 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:25.333708 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:25.334004 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:25.334009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:25.334013 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:25.334235 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:25.334708 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:25.334715 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:25.335088 (+ 373us) client_negotiation.cc:769] Sending connection context
0812 07:09:25.335367 (+ 279us) client_negotiation.cc:240] Negotiation successful
0812 07:09:25.335825 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":264,"threads_started":1}
I20260812 07:09:25.338675 22685 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37595
I20260812 07:09:25.338982 22685 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:25.339844 22685 heartbeater.cc:511] Master 127.21.181.126:37595 requested a full tablet report, sending...
I20260812 07:09:25.342432 22320 ts_manager.cc:194] Registered new tserver with Master: dab82c8f71884b1ab12684ac38879928 (127.21.181.65:43639)
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
I20260812 07:09:25.344105 22320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:33241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:25.364490 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37595
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:25.680966 22697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:25.681211 22697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:25.681344 22697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:25.688311 22697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:25.688405 22697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:25.688570 22697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:25.697777 22697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:25.698869 22697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:25.700122 22697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:25.712204 22703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:25.712345 22702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:25.712237 22708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:25.713598 22697 server_base.cc:1034] running on GCE node
I20260812 07:09:25.714126 22697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:25.714951 22697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:25.716169 22697 hybrid_clock.cc:648] HybridClock initialized: now 1786518565716123 us; error 29 us; skew 500 ppm
I20260812 07:09:25.716506 22697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:25 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:25.719347 22697 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:25.720685 22697 webserver.cc:460] Webserver started at http://127.21.181.66:44687/ using document root <none> and password file <none>
I20260812 07:09:25.721292 22697 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:25.721406 22697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:25.721686 22697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:25.724399 22697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "b6175dd58bff4fe29e6aad9f57a8ff9e"
format_stamp: "Formatted at 2026-08-12 07:09:25 on dist-test-slave-lgr5"
server_key: "20066e4b147b268a91f2b73256d82300"
server_key_iv: "30877f28bd94b5bbeaf16cdac2a94380"
server_key_version: "encryptionkey@0"
I20260812 07:09:25.725154 22697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "b6175dd58bff4fe29e6aad9f57a8ff9e"
format_stamp: "Formatted at 2026-08-12 07:09:25 on dist-test-slave-lgr5"
server_key: "20066e4b147b268a91f2b73256d82300"
server_key_iv: "30877f28bd94b5bbeaf16cdac2a94380"
server_key_version: "encryptionkey@0"
I20260812 07:09:25.730078 22697 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:09:25.733264 22721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:25.734388 22697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:25.734529 22697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "b6175dd58bff4fe29e6aad9f57a8ff9e"
format_stamp: "Formatted at 2026-08-12 07:09:25 on dist-test-slave-lgr5"
server_key: "20066e4b147b268a91f2b73256d82300"
server_key_iv: "30877f28bd94b5bbeaf16cdac2a94380"
server_key_version: "encryptionkey@0"
I20260812 07:09:25.734694 22697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:25.751778 22697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:25.755761 22697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:25.756016 22697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:25.756891 22697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:25.758152 22697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:25.758231 22697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:25.758347 22697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:25.758401 22697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:25.790189 22697 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:34973
I20260812 07:09:25.790232 22926 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:34973 every 8 connection(s)
I20260812 07:09:25.791378 22697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:25.797904 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22697
I20260812 07:09:25.798086 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:25.798512 22229 external_mini_cluster.cc:1428] Setting key 0a2c44613e510ca0bbd89d187cf2092a
Aug 12 07:09:25 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518565, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:25.810110 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:25.793848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:46807 (local address 127.21.181.126:37595)
0812 07:09:25.794056 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:25.794063 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:25.795203 (+ 1140us) server_negotiation.cc:407] Connection header received
0812 07:09:25.796977 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:25.796982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:25.797060 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:25.797149 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:25.798619 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.799451 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.801128 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.801532 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.801651 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:25.804809 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:25.804814 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:25.804817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:25.804847 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:25.807273 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:25.807928 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:25.807932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:25.807934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:25.808070 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:25.808514 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:25.808519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:25.808521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:25.808763 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:25.808897 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:25.809570 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 07:09:25.809796 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:09:25.810914 22933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:25.794361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37595 (local address 127.21.181.66:46807)
0812 07:09:25.795037 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:25.795097 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:25.796719 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:25.797384 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:25.797392 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:25.797861 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:25.798457 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:25.798475 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.799604 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.799608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:25.800989 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:25.801002 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:25.801764 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:25.801772 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:25.801981 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:25.802564 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:25.802591 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:25.804606 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:25.807404 (+ 2798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:25.807413 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:25.807427 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:25.807804 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:25.808198 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:25.808203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:25.808207 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:25.808400 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:25.808889 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:25.808896 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:25.809199 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 07:09:25.809418 (+ 219us) client_negotiation.cc:240] Negotiation successful
0812 07:09:25.809870 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":255,"threads_started":1}
I20260812 07:09:25.812326 22928 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37595
I20260812 07:09:25.812608 22928 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:25.813256 22928 heartbeater.cc:511] Master 127.21.181.126:37595 requested a full tablet report, sending...
I20260812 07:09:25.814711 22320 ts_manager.cc:194] Registered new tserver with Master: b6175dd58bff4fe29e6aad9f57a8ff9e (127.21.181.66:34973)
I20260812 07:09:25.815785 22320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:46807
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:25.837298 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37595
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:26.154042 22938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:26.154314 22938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:26.154417 22938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:26.161661 22938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:26.161768 22938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:26.161888 22938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:26.171232 22938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37595
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:26.172401 22938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:26.173624 22938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:26.186271 22953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:26.186324 22961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:26.186769 22952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:26.187590 22938 server_base.cc:1034] running on GCE node
I20260812 07:09:26.188206 22938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:26.189079 22938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:26.190274 22938 hybrid_clock.cc:648] HybridClock initialized: now 1786518566190218 us; error 40 us; skew 500 ppm
I20260812 07:09:26.190572 22938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:26.193621 22938 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:26.195007 22938 webserver.cc:460] Webserver started at http://127.21.181.67:42617/ using document root <none> and password file <none>
I20260812 07:09:26.195662 22938 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:26.195777 22938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:26.196069 22938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:26.198870 22938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "44a34d9ef96f424583f5f2287acf2082"
format_stamp: "Formatted at 2026-08-12 07:09:26 on dist-test-slave-lgr5"
server_key: "6e3da19095df334cc884d981e427edde"
server_key_iv: "cda561c4abd5f43f5cab993bd2dc7e5d"
server_key_version: "encryptionkey@0"
I20260812 07:09:26.199631 22938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "44a34d9ef96f424583f5f2287acf2082"
format_stamp: "Formatted at 2026-08-12 07:09:26 on dist-test-slave-lgr5"
server_key: "6e3da19095df334cc884d981e427edde"
server_key_iv: "cda561c4abd5f43f5cab993bd2dc7e5d"
server_key_version: "encryptionkey@0"
I20260812 07:09:26.204689 22938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260812 07:09:26.207944 22973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:26.209163 22938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260812 07:09:26.209316 22938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "44a34d9ef96f424583f5f2287acf2082"
format_stamp: "Formatted at 2026-08-12 07:09:26 on dist-test-slave-lgr5"
server_key: "6e3da19095df334cc884d981e427edde"
server_key_iv: "cda561c4abd5f43f5cab993bd2dc7e5d"
server_key_version: "encryptionkey@0"
I20260812 07:09:26.209502 22938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:26.227684 22938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:26.231165 22938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:26.231423 22938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:26.232299 22938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:26.233497 22938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:26.233579 22938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:26.233697 22938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:26.233750 22938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:26.264524 22938 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:41089
I20260812 07:09:26.264573 23213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:41089 every 8 connection(s)
I20260812 07:09:26.265741 22938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:26.270807 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22938
I20260812 07:09:26.271019 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:26.271582 22229 external_mini_cluster.cc:1428] Setting key 44178bbabff51966e2aef3abce0dc7f4
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:26.286608 22688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.267979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:40771 (local address 127.21.181.126:37595)
0812 07:09:26.268175 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:26.268180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:26.269560 (+ 1380us) server_negotiation.cc:407] Connection header received
0812 07:09:26.271430 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:26.271435 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:26.271516 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:26.271604 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:26.273485 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.274660 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.276435 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.277096 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.277256 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.280842 (+ 3586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:26.280847 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:26.280851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:26.280883 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:26.283342 (+ 2459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.284216 (+ 874us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.284220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.284222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.284340 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.284790 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.284794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.284795 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.285044 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:26.285153 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:26.286007 (+ 854us) server_negotiation.cc:299] Negotiation successful
0812 07:09:26.286255 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:09:26.287750 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.268654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37595 (local address 127.21.181.67:40771)
0812 07:09:26.269393 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:26.269437 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:26.271170 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:26.271856 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:26.271864 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:26.272589 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:26.273291 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:26.273318 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.274880 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.274886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:26.276287 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:26.276302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.277325 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.277331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:26.277578 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.278170 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:26.278198 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:26.280625 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:26.283480 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:26.283492 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:26.283506 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:26.284113 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:26.284437 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:26.284442 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:26.284448 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:26.284694 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:26.285163 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:26.285171 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:26.285582 (+ 411us) client_negotiation.cc:769] Sending connection context
0812 07:09:26.285894 (+ 312us) client_negotiation.cc:240] Negotiation successful
0812 07:09:26.286417 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":314,"threads_started":1}
I20260812 07:09:26.289108 23214 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37595
I20260812 07:09:26.289381 23214 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:26.290109 23214 heartbeater.cc:511] Master 127.21.181.126:37595 requested a full tablet report, sending...
I20260812 07:09:26.291445 22320 ts_manager.cc:194] Registered new tserver with Master: 44a34d9ef96f424583f5f2287acf2082 (127.21.181.67:41089)
I20260812 07:09:26.292299 22320 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:40771
I20260812 07:09:26.299501 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:26.345422 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.330399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55836 (local address 127.21.181.65:43639)
0812 07:09:26.330958 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:26.330965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:26.331067 (+ 102us) server_negotiation.cc:407] Connection header received
0812 07:09:26.331205 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:26.331211 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:26.331398 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:26.331534 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:26.333039 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.334321 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.335547 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.336067 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.336190 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.339931 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:26.339942 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:26.339954 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:26.339987 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:26.342763 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.343382 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.343389 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.343391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.343525 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.343901 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.343905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.343907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.344267 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:26.344571 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:26.344834 (+ 263us) server_negotiation.cc:299] Negotiation successful
0812 07:09:26.345081 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":266,"threads_started":1}
W20260812 07:09:26.346572 22528 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55836
I20260812 07:09:26.346815 22685 heartbeater.cc:503] Master 127.21.181.126:37595 was elected leader, sending a full tablet report...
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:26.389645 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.377224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55842 (local address 127.21.181.65:43639)
0812 07:09:26.377463 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:26.377471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:26.377626 (+ 155us) server_negotiation.cc:407] Connection header received
0812 07:09:26.377801 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:26.377804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:26.377895 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:26.377971 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:26.379212 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.379885 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.381051 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.381439 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.381528 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.384925 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:26.384930 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:26.384933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:26.384988 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:26.387206 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.387810 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.387816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.387818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.387938 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.388342 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.388345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.388347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.388617 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:26.388844 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:26.389147 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 07:09:26.389352 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:09:26.391016 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22451
I20260812 07:09:26.402563 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.391956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55854 (local address 127.21.181.65:43639)
0812 07:09:26.392157 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:26.392161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:26.392384 (+ 223us) server_negotiation.cc:407] Connection header received
0812 07:09:26.392515 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:26.392521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:26.392578 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:26.392650 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:26.393764 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.394403 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.395592 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.395992 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.396117 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.398088 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:26.398093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:26.398098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:26.398125 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:26.400288 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.400862 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.400867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.400871 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.401003 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.401326 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.401331 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.401333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.401568 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:26.401771 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:26.402057 (+ 286us) server_negotiation.cc:299] Negotiation successful
0812 07:09:26.402242 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:09:26.403195 22677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:26.483551 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22451
I20260812 07:09:26.514278 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22697
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:09:26.529320 23239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.515172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.21.181.66:34973)
0812 07:09:26.515642 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:26.515649 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:26.515675 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:09:26.515735 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:26.515741 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:26.515949 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:26.516113 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:26.517233 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.518218 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.519357 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.519868 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.519960 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.523722 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:26.523733 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:26.523747 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:26.523778 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:26.526513 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.527124 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.527131 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.527133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.527257 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.527651 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.527655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.527657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.527987 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:26.528315 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:26.528733 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 07:09:26.529010 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":222,"threads_started":1}
I20260812 07:09:26.530470 22920 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:26.609797 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22697
I20260812 07:09:26.630484 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22938
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[22239](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:09:26.645787 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:26.631447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59502 (local address 127.21.181.67:41089)
0812 07:09:26.632073 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:26.632106 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:26.632140 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:09:26.632202 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:26.632208 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:26.632410 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:26.632563 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:26.633801 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.634828 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.636119 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:26.636671 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:26.636787 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:26.640591 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:26.640603 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:26.640616 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:26.640652 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:26.643113 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.643727 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.643734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.643736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.643861 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:26.644237 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:26.644241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:26.644243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:26.644568 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:26.644849 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:26.645163 (+ 314us) server_negotiation.cc:299] Negotiation successful
0812 07:09:26.645418 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":312,"threads_started":1}
I20260812 07:09:26.646960 23208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:26.727237 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22938
I20260812 07:09:26.747498 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22259
I20260812 07:09:26.748606 22382 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:26.848660 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22259
2026-08-12T07:09:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4010 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[23249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[23249](info): set up 2 sockets
Aug 12 07:09:26 dist-test-slave-lgr5 krb5kdc[23249](info): commencing operation
krb5kdc: starting...
W20260812 07:09:28.078842 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.007s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:28 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:28.246590 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38673
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:38673
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:28.554265 23276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:28.554544 23276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:28.554674 23276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:28.561631 23276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:28.561734 23276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:28.561810 23276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:28.561888 23276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:28.561947 23276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:28.571223 23276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:38673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:28.572397 23276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:28.573608 23276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:28.584492 23287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:28.584499 23285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:28.584525 23296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:28.585466 23276 server_base.cc:1034] running on GCE node
I20260812 07:09:28.586171 23276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:28.587378 23276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:28.588565 23276 hybrid_clock.cc:648] HybridClock initialized: now 1786518568588513 us; error 39 us; skew 500 ppm
I20260812 07:09:28.588858 23276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:28 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518568, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:28.591697 23276 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:28.593062 23276 webserver.cc:460] Webserver started at http://127.21.181.126:44541/ using document root <none> and password file <none>
I20260812 07:09:28.593701 23276 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:28.593806 23276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:28.594105 23276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:28.596917 23276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "07921c4c4e014ace9e7b3b9d3928c78d"
format_stamp: "Formatted at 2026-08-12 07:09:28 on dist-test-slave-lgr5"
server_key: "427bf30c6778c56a7f6342c721ff53f9"
server_key_iv: "95657b3e8e205744fde90d9f1b99353a"
server_key_version: "encryptionkey@0"
I20260812 07:09:28.597654 23276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "07921c4c4e014ace9e7b3b9d3928c78d"
format_stamp: "Formatted at 2026-08-12 07:09:28 on dist-test-slave-lgr5"
server_key: "427bf30c6778c56a7f6342c721ff53f9"
server_key_iv: "95657b3e8e205744fde90d9f1b99353a"
server_key_version: "encryptionkey@0"
I20260812 07:09:28.602392 23276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:28.605557 23310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:28.606709 23276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:28.606861 23276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "07921c4c4e014ace9e7b3b9d3928c78d"
format_stamp: "Formatted at 2026-08-12 07:09:28 on dist-test-slave-lgr5"
server_key: "427bf30c6778c56a7f6342c721ff53f9"
server_key_iv: "95657b3e8e205744fde90d9f1b99353a"
server_key_version: "encryptionkey@0"
I20260812 07:09:28.607026 23276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:28.626559 23276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:28.630053 23276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:28.630292 23276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:28.646517 23276 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:38673
I20260812 07:09:28.646523 23428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:38673 every 8 connection(s)
I20260812 07:09:28.647761 23276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:28.649194 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 23276
I20260812 07:09:28.649425 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:28.650072 22229 external_mini_cluster.cc:1428] Setting key 6851d9264d52ef40554968ed0bd579d3
I20260812 07:09:28.651803 23430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:09:28 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:28.664891 23430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d: Bootstrap starting.
I20260812 07:09:28.668120 23430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:28.669296 23430 log.cc:824] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:28.672459 23430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d: No bootstrap required, opened a new log
I20260812 07:09:28.672590 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:28.651856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37088 (local address 127.21.181.126:38673)
0812 07:09:28.652701 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:28.652717 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:28.652769 (+ 52us) server_negotiation.cc:407] Connection header received
0812 07:09:28.654811 (+ 2042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:28.654838 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:28.655321 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:28.655756 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:28.657090 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:28.659072 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:28.660250 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:28.661211 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:28.661377 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:28.665435 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:28.665443 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:28.665455 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:28.665494 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:28.668321 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:28.669074 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:28.669080 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:28.669086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:28.669237 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:28.669623 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:28.669626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:28.669628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:28.670160 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:28.670395 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:28.670942 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 07:09:28.671346 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":356,"threads_started":1}
I20260812 07:09:28.680480 23430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38673 } }
I20260812 07:09:28.680785 23430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:28.680855 23430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07921c4c4e014ace9e7b3b9d3928c78d, State: Initialized, Role: FOLLOWER
I20260812 07:09:28.681298 23430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38673 } }
I20260812 07:09:28.681491 23430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:28.681586 23430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:28.681728 23430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:28.684005 23430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38673 } }
I20260812 07:09:28.684455 23430 leader_election.cc:302] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07921c4c4e014ace9e7b3b9d3928c78d; no voters:
I20260812 07:09:28.684953 23430 leader_election.cc:288] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:28.685073 23440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:28.685698 23440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [term 1 LEADER]: Becoming Leader. State: Replica: 07921c4c4e014ace9e7b3b9d3928c78d, State: Running, Role: LEADER
I20260812 07:09:28.686194 23440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38673 } }
I20260812 07:09:28.686774 23430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:28.688642 23442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38673 } } }
I20260812 07:09:28.689036 23442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:28.689477 23449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07921c4c4e014ace9e7b3b9d3928c78d. Latest consensus state: current_term: 1 leader_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07921c4c4e014ace9e7b3b9d3928c78d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38673 } } }
I20260812 07:09:28.689790 23449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:28.690856 23460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:28.694931 23460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:28.702729 23460 catalog_manager.cc:1347] Generated new cluster ID: ccb6f44ae8d44b3cb8823c010567b4e9
I20260812 07:09:28.702824 23460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:28.718358 23460 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:28.719498 23460 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:28.729314 23460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07921c4c4e014ace9e7b3b9d3928c78d: Generated new TSK 0
I20260812 07:09:28.729993 23460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:28.777889 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38673
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:29.092478 23484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:29.092803 23484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:29.092943 23484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:29.099947 23484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:29.100063 23484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:29.100255 23484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:29.109782 23484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38673
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:29.110957 23484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:29.112254 23484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:29.124480 23507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:29.124527 23504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:29.124713 23503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:29.125356 23484 server_base.cc:1034] running on GCE node
I20260812 07:09:29.125900 23484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:29.126746 23484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:29.127945 23484 hybrid_clock.cc:648] HybridClock initialized: now 1786518569127904 us; error 26 us; skew 500 ppm
I20260812 07:09:29.128262 23484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:29 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:29.131271 23484 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:29.132616 23484 webserver.cc:460] Webserver started at http://127.21.181.65:35039/ using document root <none> and password file <none>
I20260812 07:09:29.133426 23484 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:29.133570 23484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:29.133949 23484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:29.137914 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "924312cb96f04662943d4bc9f6aedb74"
format_stamp: "Formatted at 2026-08-12 07:09:29 on dist-test-slave-lgr5"
server_key: "620c29d86b696a56f5c4fec387f49cac"
server_key_iv: "d43d74d22c84119a318fa7eb0929206f"
server_key_version: "encryptionkey@0"
I20260812 07:09:29.138929 23484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "924312cb96f04662943d4bc9f6aedb74"
format_stamp: "Formatted at 2026-08-12 07:09:29 on dist-test-slave-lgr5"
server_key: "620c29d86b696a56f5c4fec387f49cac"
server_key_iv: "d43d74d22c84119a318fa7eb0929206f"
server_key_version: "encryptionkey@0"
I20260812 07:09:29.145852 23484 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260812 07:09:29.149390 23520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:29.150710 23484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:09:29.150880 23484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "924312cb96f04662943d4bc9f6aedb74"
format_stamp: "Formatted at 2026-08-12 07:09:29 on dist-test-slave-lgr5"
server_key: "620c29d86b696a56f5c4fec387f49cac"
server_key_iv: "d43d74d22c84119a318fa7eb0929206f"
server_key_version: "encryptionkey@0"
I20260812 07:09:29.151064 23484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:29.168442 23484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:29.172395 23484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:29.172639 23484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:29.173532 23484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:29.174816 23484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:29.174901 23484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:29.175017 23484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:29.175082 23484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:29.205452 23484 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:41555
I20260812 07:09:29.205525 23728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:41555 every 8 connection(s)
I20260812 07:09:29.206612 23484 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:29.214064 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 23484
I20260812 07:09:29.214258 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:29.214849 22229 external_mini_cluster.cc:1428] Setting key 482603f24143407cdfeed4e9addeb686
Aug 12 07:09:29 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:29.225419 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:29.209028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:58531 (local address 127.21.181.126:38673)
0812 07:09:29.209453 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:29.209458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:29.210441 (+ 983us) server_negotiation.cc:407] Connection header received
0812 07:09:29.212551 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:29.212557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:29.212635 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:29.212730 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:29.214227 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.214988 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.216536 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.216938 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.217046 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:29.220266 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:29.220271 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:29.220274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:29.220304 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:29.222590 (+ 2286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:29.223247 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:29.223253 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:29.223255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:29.223369 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:29.223768 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:29.223774 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:29.223777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:29.224054 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:29.224215 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:29.224849 (+ 634us) server_negotiation.cc:299] Negotiation successful
0812 07:09:29.225066 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":209,"threads_started":1}
I20260812 07:09:29.226279 23737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:29.209528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38673 (local address 127.21.181.65:58531)
0812 07:09:29.210267 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:29.210329 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:29.212278 (+ 1949us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:29.212970 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:29.212978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:29.213512 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:29.214059 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:29.214076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.215144 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.215149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:29.216398 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:29.216411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.217159 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.217167 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:29.217393 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:29.217988 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:29.218013 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:29.220036 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:29.222716 (+ 2680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:29.222725 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:29.222739 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:29.223126 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:29.223469 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:29.223474 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:29.223478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:29.223656 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:29.224186 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:29.224194 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:29.224505 (+ 311us) client_negotiation.cc:769] Sending connection context
0812 07:09:29.224722 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:09:29.225188 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":279,"threads_started":1}
I20260812 07:09:29.227684 23729 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38673
I20260812 07:09:29.227948 23729 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:29.228703 23729 heartbeater.cc:511] Master 127.21.181.126:38673 requested a full tablet report, sending...
I20260812 07:09:29.230997 23355 ts_manager.cc:194] Registered new tserver with Master: 924312cb96f04662943d4bc9f6aedb74 (127.21.181.65:41555)
I20260812 07:09:29.233091 23355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:58531
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:29.254899 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38673
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:29.564718 23746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:29.564976 23746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:29.565110 23746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:29.571959 23746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:29.572063 23746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:29.572232 23746 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:29.581297 23746 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38673
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:29.582410 23746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:29.583606 23746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:29.596223 23763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:29.596180 23772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:29.596251 23766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:29.596980 23746 server_base.cc:1034] running on GCE node
I20260812 07:09:29.597497 23746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:29.598343 23746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:29.599535 23746 hybrid_clock.cc:648] HybridClock initialized: now 1786518569599488 us; error 42 us; skew 500 ppm
I20260812 07:09:29.599812 23746 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:29 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:29.602715 23746 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:29.603981 23746 webserver.cc:460] Webserver started at http://127.21.181.66:34955/ using document root <none> and password file <none>
I20260812 07:09:29.604617 23746 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:29.604732 23746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:29.605024 23746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:29.607668 23746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "e076fe4773b9475a987557fd6ec5c82e"
format_stamp: "Formatted at 2026-08-12 07:09:29 on dist-test-slave-lgr5"
server_key: "3a1dadce66c024d001146f2d2af4f3cb"
server_key_iv: "1af782c4615e28a56734b5eb3b88e787"
server_key_version: "encryptionkey@0"
I20260812 07:09:29.608474 23746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "e076fe4773b9475a987557fd6ec5c82e"
format_stamp: "Formatted at 2026-08-12 07:09:29 on dist-test-slave-lgr5"
server_key: "3a1dadce66c024d001146f2d2af4f3cb"
server_key_iv: "1af782c4615e28a56734b5eb3b88e787"
server_key_version: "encryptionkey@0"
I20260812 07:09:29.613202 23746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:09:29.616395 23791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:29.617552 23746 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:29.617698 23746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "e076fe4773b9475a987557fd6ec5c82e"
format_stamp: "Formatted at 2026-08-12 07:09:29 on dist-test-slave-lgr5"
server_key: "3a1dadce66c024d001146f2d2af4f3cb"
server_key_iv: "1af782c4615e28a56734b5eb3b88e787"
server_key_version: "encryptionkey@0"
I20260812 07:09:29.617863 23746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:29.632669 23746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:29.636391 23746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:29.636623 23746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:29.637460 23746 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:29.638623 23746 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:29.638700 23746 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:29.638818 23746 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:29.638870 23746 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:29.668663 23746 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:42575
I20260812 07:09:29.668721 24002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:42575 every 8 connection(s)
I20260812 07:09:29.669796 23746 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:29.679649 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 23746
I20260812 07:09:29.679827 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:29.680311 22229 external_mini_cluster.cc:1428] Setting key 103787e44cea0efa2b3e450700ded9e1
Aug 12 07:09:29 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518569, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:29.688308 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:29.672150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:44577 (local address 127.21.181.126:38673)
0812 07:09:29.672344 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:29.672350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:29.673467 (+ 1117us) server_negotiation.cc:407] Connection header received
0812 07:09:29.675174 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:29.675178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:29.675244 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:29.675329 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:29.676731 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.677462 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.678914 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.679367 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.679496 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:29.682887 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:29.682892 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:29.682896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:29.682927 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:29.685328 (+ 2401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:29.686105 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:29.686108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:29.686110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:29.686238 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:29.686673 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:29.686677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:29.686678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:29.686968 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:29.687086 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:29.687824 (+ 738us) server_negotiation.cc:299] Negotiation successful
0812 07:09:29.688029 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:09:29.689302 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:29.672619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38673 (local address 127.21.181.66:44577)
0812 07:09:29.673298 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:29.673352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:29.674930 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:29.675504 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:29.675512 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:29.676001 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:29.676580 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:29.676598 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.677610 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.677614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:29.678792 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:29.678804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:29.679541 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:29.679546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:29.679725 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:29.680392 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:29.680420 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:29.682669 (+ 2249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:29.685461 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:29.685469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:29.685482 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:29.686003 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:29.686337 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:29.686341 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:29.686344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:29.686565 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:29.687076 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:29.687082 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:29.687447 (+ 365us) client_negotiation.cc:769] Sending connection context
0812 07:09:29.687717 (+ 270us) client_negotiation.cc:240] Negotiation successful
0812 07:09:29.688201 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":264,"threads_started":1}
I20260812 07:09:29.690557 24003 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38673
I20260812 07:09:29.690852 24003 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:29.691648 24003 heartbeater.cc:511] Master 127.21.181.126:38673 requested a full tablet report, sending...
I20260812 07:09:29.693116 23355 ts_manager.cc:194] Registered new tserver with Master: e076fe4773b9475a987557fd6ec5c82e (127.21.181.66:42575)
I20260812 07:09:29.693905 23355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:44577
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:29.720860 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38673
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:30.030576 24009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:30.030849 24009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:30.030989 24009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:30.038034 24009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:30.038144 24009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:30.038295 24009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:30.047544 24009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38673
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:30.048830 24009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:30.050163 24009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:30.062420 24020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:30.062458 24024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:30.062556 24021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:30.063056 24009 server_base.cc:1034] running on GCE node
I20260812 07:09:30.063642 24009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:30.064540 24009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:30.065737 24009 hybrid_clock.cc:648] HybridClock initialized: now 1786518570065691 us; error 36 us; skew 500 ppm
I20260812 07:09:30.066044 24009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:30.068912 24009 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:30.070250 24009 webserver.cc:460] Webserver started at http://127.21.181.67:33651/ using document root <none> and password file <none>
I20260812 07:09:30.070868 24009 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:30.070983 24009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:30.071274 24009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:30.074102 24009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "388565fc6d5a49b69add82034ff005de"
format_stamp: "Formatted at 2026-08-12 07:09:30 on dist-test-slave-lgr5"
server_key: "3d19db9a52153658130522f659aa45f2"
server_key_iv: "90ccebf2402753778545124ad806db73"
server_key_version: "encryptionkey@0"
I20260812 07:09:30.074833 24009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "388565fc6d5a49b69add82034ff005de"
format_stamp: "Formatted at 2026-08-12 07:09:30 on dist-test-slave-lgr5"
server_key: "3d19db9a52153658130522f659aa45f2"
server_key_iv: "90ccebf2402753778545124ad806db73"
server_key_version: "encryptionkey@0"
I20260812 07:09:30.079516 24009 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:09:30.083618 24035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:30.084870 24009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:09:30.085026 24009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "388565fc6d5a49b69add82034ff005de"
format_stamp: "Formatted at 2026-08-12 07:09:30 on dist-test-slave-lgr5"
server_key: "3d19db9a52153658130522f659aa45f2"
server_key_iv: "90ccebf2402753778545124ad806db73"
server_key_version: "encryptionkey@0"
I20260812 07:09:30.085201 24009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:30.107271 24009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:30.110828 24009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:30.111085 24009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:30.111970 24009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:30.113215 24009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:30.113292 24009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:30.113416 24009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:30.113479 24009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:30.143755 24009 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:43601
I20260812 07:09:30.143810 24225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:43601 every 8 connection(s)
I20260812 07:09:30.145177 24009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:30.155134 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24009
I20260812 07:09:30.155316 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:30.155916 22229 external_mini_cluster.cc:1428] Setting key 1733f1b0783f1c72392f08dc73806fd8
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:30.164880 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.147829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:42797 (local address 127.21.181.126:38673)
0812 07:09:30.148032 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.148038 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.149260 (+ 1222us) server_negotiation.cc:407] Connection header received
0812 07:09:30.151090 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.151094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.151161 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.151247 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.152862 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.153614 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.155360 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.156004 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.156166 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.159995 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.160001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.160004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.160038 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.162388 (+ 2350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.163041 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.163044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.163046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.163154 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.163488 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.163491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.163493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.163742 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.163848 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.164461 (+ 613us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.164629 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:09:30.165817 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.148404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38673 (local address 127.21.181.67:42797)
0812 07:09:30.149107 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:30.149147 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:30.150830 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:30.151460 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:30.151467 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:30.152073 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:30.152700 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.152718 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.153761 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.153765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.155153 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.155166 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.156282 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.156292 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.156611 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.157281 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:30.157305 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:30.159791 (+ 2486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:30.162526 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:30.162533 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:30.162545 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:30.162954 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:30.163227 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:30.163230 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:30.163232 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:30.163406 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:30.163833 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:30.163839 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:30.164157 (+ 318us) client_negotiation.cc:769] Sending connection context
0812 07:09:30.164382 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:09:30.164755 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":270,"threads_started":1}
I20260812 07:09:30.167078 24227 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38673
I20260812 07:09:30.167351 24227 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:30.168030 24227 heartbeater.cc:511] Master 127.21.181.126:38673 requested a full tablet report, sending...
I20260812 07:09:30.169376 23355 ts_manager.cc:194] Registered new tserver with Master: 388565fc6d5a49b69add82034ff005de (127.21.181.67:43601)
I20260812 07:09:30.170301 23355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:42797
I20260812 07:09:30.171260 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:09:30.189082 23735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.177567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.21.181.126:38673)
0812 07:09:30.177760 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.177766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.178146 (+ 380us) server_negotiation.cc:407] Connection header received
0812 07:09:30.178271 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.178274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.178340 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.178415 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.179770 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.180520 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.181735 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.182193 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.182284 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.184394 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.184399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.184402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.184434 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.186799 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.187356 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.187359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.187362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.187480 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.187787 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.187790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.187792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.188134 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.188318 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.188642 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.188801 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:09:30.194895 23355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:30.202029 23355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:30.235906 23729 heartbeater.cc:503] Master 127.21.181.126:38673 was elected leader, sending a full tablet report...
I20260812 07:09:30.236109 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.214694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.21.181.67:43601)
0812 07:09:30.227462 (+ 12768us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.227470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.227578 (+ 108us) server_negotiation.cc:407] Connection header received
0812 07:09:30.227685 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.227692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.227931 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.228168 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:30.229211 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.230834 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.233382 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.234777 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.234938 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.235206 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.235512 (+ 306us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.235688 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12608,"thread_start_us":12463,"threads_started":1}
I20260812 07:09:30.237882 24246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.214474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:43601 (local address 127.0.0.1:52002)
0812 07:09:30.225546 (+ 11072us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:30.225602 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:30.225788 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:30.228530 (+ 2742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:30.228539 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:30.228580 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:30.228986 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.228998 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.230988 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.230993 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.233195 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.233216 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.235026 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.235035 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.235260 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.235308 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 07:09:30.235438 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 07:09:30.237461 (+ 2023us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10787,"mutex_wait_us":1,"thread_start_us":15723,"threads_started":1}
I20260812 07:09:30.238763 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.227028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.21.181.66:42575)
0812 07:09:30.229816 (+ 2788us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.229824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.230654 (+ 830us) server_negotiation.cc:407] Connection header received
0812 07:09:30.230769 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.230776 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.230958 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.231160 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:30.232387 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.233499 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.235608 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.237364 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.237489 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.237678 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.238293 (+ 615us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.238458 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2641,"thread_start_us":2525,"threads_started":1}
I20260812 07:09:30.238790 24245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.228113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:42575 (local address 127.0.0.1:59588)
0812 07:09:30.230485 (+ 2372us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:30.230502 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:30.230625 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:30.231427 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:30.231433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:30.231470 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:30.232162 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.232174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.233623 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.233628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.235424 (+ 1796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.235437 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.237839 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.237845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.238112 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.238137 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:09:30.238222 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:09:30.238337 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2238,"mutex_wait_us":96,"spinlock_wait_cycles":1408,"thread_start_us":1814,"threads_started":1}
I20260812 07:09:30.240034 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.215188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59982 (local address 127.21.181.65:41555)
0812 07:09:30.227778 (+ 12590us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.227786 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.230391 (+ 2605us) server_negotiation.cc:407] Connection header received
0812 07:09:30.230724 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.230733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.230958 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.231159 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:30.232332 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.233749 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.236307 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.237653 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.237781 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.237975 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.238737 (+ 762us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.239674 (+ 937us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12434,"thread_start_us":12594,"threads_started":1}
I20260812 07:09:30.240830 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.214556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:41555 (local address 127.0.0.1:59982)
0812 07:09:30.230249 (+ 15693us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:30.230272 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:30.230504 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:30.231427 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:30.231433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:30.231483 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:30.232139 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.232155 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.233906 (+ 1751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.233913 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.236055 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.236072 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.238192 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.238199 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.238508 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.238547 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:09:30.240257 (+ 1710us) client_negotiation.cc:240] Negotiation successful
0812 07:09:30.240407 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15548,"mutex_wait_us":56,"spinlock_wait_cycles":3456,"thread_start_us":11443,"threads_started":1}
I20260812 07:09:30.241531 24120 tablet_service.cc:1467] Processing CreateTablet for tablet 03b8d794e0654ff98d6a6ab8f94856d9 (DEFAULT_TABLE table=test-table [id=3b9eb89bc1824facb4aaec234d4d666e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:30.242354 23872 tablet_service.cc:1467] Processing CreateTablet for tablet 03b8d794e0654ff98d6a6ab8f94856d9 (DEFAULT_TABLE table=test-table [id=3b9eb89bc1824facb4aaec234d4d666e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:30.243021 24120 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b8d794e0654ff98d6a6ab8f94856d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:30.243697 23872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b8d794e0654ff98d6a6ab8f94856d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:30.244325 23599 tablet_service.cc:1467] Processing CreateTablet for tablet 03b8d794e0654ff98d6a6ab8f94856d9 (DEFAULT_TABLE table=test-table [id=3b9eb89bc1824facb4aaec234d4d666e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:30.245669 23599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03b8d794e0654ff98d6a6ab8f94856d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:30.254962 24258 tablet_bootstrap.cc:492] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: Bootstrap starting.
I20260812 07:09:30.257370 24259 tablet_bootstrap.cc:492] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: Bootstrap starting.
I20260812 07:09:30.259966 24261 tablet_bootstrap.cc:492] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: Bootstrap starting.
I20260812 07:09:30.260182 24258 tablet_bootstrap.cc:654] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:30.261040 24259 tablet_bootstrap.cc:654] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:30.261639 24258 log.cc:824] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:30.262377 24259 log.cc:824] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:30.262949 24261 tablet_bootstrap.cc:654] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:30.263875 24261 log.cc:824] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:30.264770 24258 tablet_bootstrap.cc:492] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: No bootstrap required, opened a new log
I20260812 07:09:30.265041 24258 ts_tablet_manager.cc:1397] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260812 07:09:30.266187 24259 tablet_bootstrap.cc:492] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: No bootstrap required, opened a new log
I20260812 07:09:30.266430 24259 ts_tablet_manager.cc:1397] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 07:09:30.266853 24261 tablet_bootstrap.cc:492] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: No bootstrap required, opened a new log
I20260812 07:09:30.267105 24261 ts_tablet_manager.cc:1397] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:09:30.273689 24259 raft_consensus.cc:348] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.274044 24259 raft_consensus.cc:374] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:30.274144 24259 raft_consensus.cc:729] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 388565fc6d5a49b69add82034ff005de, State: Initialized, Role: FOLLOWER
I20260812 07:09:30.274688 24259 consensus_queue.cc:260] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.275744 24227 heartbeater.cc:503] Master 127.21.181.126:38673 was elected leader, sending a full tablet report...
I20260812 07:09:30.275780 24258 raft_consensus.cc:348] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.276244 24258 raft_consensus.cc:374] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:30.276355 24258 raft_consensus.cc:729] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e076fe4773b9475a987557fd6ec5c82e, State: Initialized, Role: FOLLOWER
I20260812 07:09:30.276376 24259 ts_tablet_manager.cc:1428] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: Time spent starting tablet: real 0.010s user 0.006s sys 0.005s
I20260812 07:09:30.277032 24258 consensus_queue.cc:260] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.278155 24003 heartbeater.cc:503] Master 127.21.181.126:38673 was elected leader, sending a full tablet report...
I20260812 07:09:30.278009 24261 raft_consensus.cc:348] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.278447 24261 raft_consensus.cc:374] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:30.278561 24261 raft_consensus.cc:729] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 924312cb96f04662943d4bc9f6aedb74, State: Initialized, Role: FOLLOWER
I20260812 07:09:30.279241 24261 consensus_queue.cc:260] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.280339 24258 ts_tablet_manager.cc:1428] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260812 07:09:30.282941 24261 ts_tablet_manager.cc:1428] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260812 07:09:30.387252 24269 raft_consensus.cc:482] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:30.387537 24269 raft_consensus.cc:504] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.389398 24269 leader_election.cc:288] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 924312cb96f04662943d4bc9f6aedb74 (127.21.181.65:41555), e076fe4773b9475a987557fd6ec5c82e (127.21.181.66:42575)
I20260812 07:09:30.395514 24230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.389589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:41555 (local address 127.21.181.67:47989)
0812 07:09:30.389779 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:30.389800 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:30.389947 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:30.390388 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:30.390393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:30.390425 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:30.390645 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.390658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.391895 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.391898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.393714 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.393729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.394966 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.394973 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.395131 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.395156 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:09:30.395226 (+ 70us) client_negotiation.cc:240] Negotiation successful
0812 07:09:30.395318 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":19}
I20260812 07:09:30.395640 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.389728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:47989 (local address 127.21.181.65:41555)
0812 07:09:30.389949 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.389956 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.389977 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:09:30.390083 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.390088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.390169 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.390262 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:30.390799 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.391765 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.393860 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.394797 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.394949 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.395034 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.395275 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.395402 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:09:30.396620 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.390095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:45003 (local address 127.21.181.66:42575)
0812 07:09:30.390283 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.390289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.390755 (+ 466us) server_negotiation.cc:407] Connection header received
0812 07:09:30.390889 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.390894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.390971 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.391069 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:30.391506 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.392371 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.394433 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.395533 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.395676 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.395743 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.396255 (+ 512us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.396373 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:09:30.396612 23641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b8d794e0654ff98d6a6ab8f94856d9" candidate_uuid: "388565fc6d5a49b69add82034ff005de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924312cb96f04662943d4bc9f6aedb74" is_pre_election: true
W20260812 07:09:30.396963 24228 tablet.cc:2369] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:30.397173 23641 raft_consensus.cc:2454] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388565fc6d5a49b69add82034ff005de in term 0.
I20260812 07:09:30.397753 24275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.389939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:42575 (local address 127.21.181.67:45003)
0812 07:09:30.390640 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:30.390656 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:30.390777 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:30.391073 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:30.391075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:30.391093 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:30.391370 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.391377 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.392525 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.392531 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.394287 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:30.394298 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.395891 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.395898 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:30.396069 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.396123 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 07:09:30.396210 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:09:30.396328 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"mutex_wait_us":1,"thread_start_us":346,"threads_started":1}
I20260812 07:09:30.397817 24039 leader_election.cc:302] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388565fc6d5a49b69add82034ff005de, 924312cb96f04662943d4bc9f6aedb74; no voters:
I20260812 07:09:30.398154 24269 raft_consensus.cc:2781] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:30.398307 24269 raft_consensus.cc:482] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:30.398413 24269 raft_consensus.cc:3037] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:30.398821 23911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b8d794e0654ff98d6a6ab8f94856d9" candidate_uuid: "388565fc6d5a49b69add82034ff005de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e076fe4773b9475a987557fd6ec5c82e" is_pre_election: true
I20260812 07:09:30.399302 23911 raft_consensus.cc:2454] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 388565fc6d5a49b69add82034ff005de in term 0.
I20260812 07:09:30.400763 24269 raft_consensus.cc:504] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.401284 24269 leader_election.cc:288] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [CANDIDATE]: Term 1 election: Requested vote from peers 924312cb96f04662943d4bc9f6aedb74 (127.21.181.65:41555), e076fe4773b9475a987557fd6ec5c82e (127.21.181.66:42575)
I20260812 07:09:30.401670 23641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b8d794e0654ff98d6a6ab8f94856d9" candidate_uuid: "388565fc6d5a49b69add82034ff005de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "924312cb96f04662943d4bc9f6aedb74"
I20260812 07:09:30.401713 23911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03b8d794e0654ff98d6a6ab8f94856d9" candidate_uuid: "388565fc6d5a49b69add82034ff005de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e076fe4773b9475a987557fd6ec5c82e"
I20260812 07:09:30.401829 23641 raft_consensus.cc:3037] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:30.401875 23911 raft_consensus.cc:3037] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:30.404016 23641 raft_consensus.cc:2454] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388565fc6d5a49b69add82034ff005de in term 1.
I20260812 07:09:30.404019 23911 raft_consensus.cc:2454] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 388565fc6d5a49b69add82034ff005de in term 1.
I20260812 07:09:30.404379 24039 leader_election.cc:302] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 388565fc6d5a49b69add82034ff005de, 924312cb96f04662943d4bc9f6aedb74; no voters:
I20260812 07:09:30.404634 24269 raft_consensus.cc:2781] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:30.405045 24269 raft_consensus.cc:686] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [term 1 LEADER]: Becoming Leader. State: Replica: 388565fc6d5a49b69add82034ff005de, State: Running, Role: LEADER
I20260812 07:09:30.405745 24269 consensus_queue.cc:237] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } }
I20260812 07:09:30.410306 23354 catalog_manager.cc:5495] T 03b8d794e0654ff98d6a6ab8f94856d9 P 388565fc6d5a49b69add82034ff005de reported cstate change: term changed from 0 to 1, leader changed from <none> to 388565fc6d5a49b69add82034ff005de (127.21.181.67). New cstate: current_term: 1 leader_uuid: "388565fc6d5a49b69add82034ff005de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "388565fc6d5a49b69add82034ff005de" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 43601 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "924312cb96f04662943d4bc9f6aedb74" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 41555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e076fe4773b9475a987557fd6ec5c82e" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42575 } health_report { overall_health: UNKNOWN } } }
W20260812 07:09:30.421536 24004 tablet.cc:2369] T 03b8d794e0654ff98d6a6ab8f94856d9 P e076fe4773b9475a987557fd6ec5c82e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:09:30.458767 23734 tablet.cc:2369] T 03b8d794e0654ff98d6a6ab8f94856d9 P 924312cb96f04662943d4bc9f6aedb74: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:30.468276 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.454409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.21.181.65:41555)
0812 07:09:30.454577 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.454584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.455043 (+ 459us) server_negotiation.cc:407] Connection header received
0812 07:09:30.455224 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.455229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.455291 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.455374 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.456882 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.457650 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.458914 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.459544 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.459637 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.463329 (+ 3692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.463342 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.463355 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.463386 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.466035 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.466599 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.466605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.466608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.466759 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.467092 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.467095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.467097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.467431 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.467653 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.467837 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.467987 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:30.515166 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.502249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60004 (local address 127.21.181.65:41555)
0812 07:09:30.502459 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.502466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.502749 (+ 283us) server_negotiation.cc:407] Connection header received
0812 07:09:30.502931 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.502937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.503005 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.503091 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.504411 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.505174 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.506413 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.506833 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.506943 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.510532 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.510537 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.510540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.510571 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.512876 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.513421 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.513426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.513430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.513559 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.513900 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.513903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.513905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.514150 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.514409 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.514641 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.514849 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:30.559196 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.546215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60012 (local address 127.21.181.65:41555)
0812 07:09:30.546398 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.546404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.546707 (+ 303us) server_negotiation.cc:407] Connection header received
0812 07:09:30.546869 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.546873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.546934 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.547015 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.548326 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.549002 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.550162 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.550589 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.550696 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.554238 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.554243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.554246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.554293 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.556784 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.557380 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.557385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.557388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.557514 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.557858 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.557861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.557863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.558157 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.558384 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.558653 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.558871 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260812 07:09:30.560887 23569 tablet_service.cc:2908] Rejecting scan request for tablet 03b8d794e0654ff98d6a6ab8f94856d9: Uninitialized: clean time has not yet been initialized
I20260812 07:09:30.563499 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 23484
I20260812 07:09:30.574259 24252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.564241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.21.181.65:41555)
0812 07:09:30.564414 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.564420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.564730 (+ 310us) server_negotiation.cc:407] Connection header received
0812 07:09:30.564828 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.564831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.564894 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.564966 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.565951 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.566578 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.567708 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.568135 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.568238 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.570034 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.570039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.570042 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.570069 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.572346 (+ 2277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.572878 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.572880 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.572882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.572979 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.573228 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.573230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.573232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.573478 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.573672 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.573842 (+ 170us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.573976 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:09:30.574817 23724 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:30.661934 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 23484
I20260812 07:09:30.691210 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 23746
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:09:30.704200 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.692024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59598 (local address 127.21.181.66:42575)
0812 07:09:30.692208 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.692214 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.692234 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:09:30.692334 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.692338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.692394 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.692478 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.693569 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.694328 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.695466 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.695820 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.695903 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.699559 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.699570 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.699583 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.699613 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.702028 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.702597 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.702604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.702606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.702717 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.702968 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.702971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.702973 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.703314 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.703544 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.703706 (+ 162us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.703859 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 07:09:30.704841 23998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:30.789965 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 23746
I20260812 07:09:30.815191 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24009
Aug 12 07:09:30 dist-test-slave-lgr5 krb5kdc[23249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:09:30.829627 24308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:30.815991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52008 (local address 127.21.181.67:43601)
0812 07:09:30.816512 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:30.816516 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:30.816532 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:09:30.816585 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:30.816589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:30.816644 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:30.816737 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:30.818014 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.818723 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.819995 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:30.820529 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:30.820655 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:30.824591 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:30.824607 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:30.824625 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:30.824677 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:30.827126 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.827729 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.827736 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.827738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.827883 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:30.828300 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:30.828303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:30.828305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:30.828644 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:30.828862 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:30.829091 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 07:09:30.829265 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":270,"threads_started":1}
I20260812 07:09:30.830238 24221 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:30.916313 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24009
I20260812 07:09:30.935340 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 23276
I20260812 07:09:30.936446 23416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:31.041462 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 23276
2026-08-12T07:09:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4196 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:31 dist-test-slave-lgr5 krb5kdc[24316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:31 dist-test-slave-lgr5 krb5kdc[24316](info): set up 2 sockets
Aug 12 07:09:31 dist-test-slave-lgr5 krb5kdc[24316](info): commencing operation
krb5kdc: starting...
W20260812 07:09:32.285463 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:32 dist-test-slave-lgr5 krb5kdc[24316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:32.453698 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:32881
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:32881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:32.757835 24339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:32.758088 24339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:32.758181 24339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:32.765069 24339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:32.765169 24339 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:09:32.765224 24339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:32.765285 24339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:32.765331 24339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:32.765398 24339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:32.774746 24339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:32881
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:32881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:32.775874 24339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:32.777096 24339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:32.787937 24350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:32.788007 24345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:32.787910 24346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:32.788592 24339 server_base.cc:1034] running on GCE node
I20260812 07:09:32.789250 24339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:32.790473 24339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:32.791684 24339 hybrid_clock.cc:648] HybridClock initialized: now 1786518572791627 us; error 35 us; skew 500 ppm
I20260812 07:09:32.791981 24339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:32 dist-test-slave-lgr5 krb5kdc[24316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:32.794785 24339 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:32.796133 24339 webserver.cc:460] Webserver started at http://127.21.181.126:37725/ using document root <none> and password file <none>
I20260812 07:09:32.796761 24339 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:32.796864 24339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:32.797159 24339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:32.799876 24339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "4e420af8532946f68cc893a598e70fd2"
format_stamp: "Formatted at 2026-08-12 07:09:32 on dist-test-slave-lgr5"
server_key: "e6fcfde6e944f1be5cb9743040bb1d4a"
server_key_iv: "6fc1dde28585731cc53da51bbd4831c3"
server_key_version: "encryptionkey@0"
I20260812 07:09:32.800652 24339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "4e420af8532946f68cc893a598e70fd2"
format_stamp: "Formatted at 2026-08-12 07:09:32 on dist-test-slave-lgr5"
server_key: "e6fcfde6e944f1be5cb9743040bb1d4a"
server_key_iv: "6fc1dde28585731cc53da51bbd4831c3"
server_key_version: "encryptionkey@0"
I20260812 07:09:32.805353 24339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:09:32.808487 24362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:32.809689 24339 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260812 07:09:32.809852 24339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "4e420af8532946f68cc893a598e70fd2"
format_stamp: "Formatted at 2026-08-12 07:09:32 on dist-test-slave-lgr5"
server_key: "e6fcfde6e944f1be5cb9743040bb1d4a"
server_key_iv: "6fc1dde28585731cc53da51bbd4831c3"
server_key_version: "encryptionkey@0"
I20260812 07:09:32.810035 24339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:32.829010 24339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:32.832479 24339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:32.832700 24339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:32.851509 24339 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:32881
I20260812 07:09:32.851538 24467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:32881 every 8 connection(s)
I20260812 07:09:32.852695 24339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:32.856294 24469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:32.856269 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24339
I20260812 07:09:32.856513 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:32.857345 22229 external_mini_cluster.cc:1428] Setting key ccd6d7ccc36edb9476935e1a6a913760
I20260812 07:09:32.867966 24469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: Bootstrap starting.
Aug 12 07:09:32 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:32.871170 24469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:32.872253 24469 log.cc:824] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:32.875531 24469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: No bootstrap required, opened a new log
I20260812 07:09:32.879726 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:32.859311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.21.181.126:32881)
0812 07:09:32.860031 (+ 720us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:32.860041 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:32.860103 (+ 62us) server_negotiation.cc:407] Connection header received
0812 07:09:32.862308 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:32.862329 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:32.862692 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:32.863026 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:32.864426 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:32.865687 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:32.866886 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:32.867557 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:32.867743 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:32.871833 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:32.871849 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:32.871862 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:32.871906 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:32.875425 (+ 3519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:32.876130 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:32.876139 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:32.876147 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:32.876367 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:32.876705 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:32.876710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:32.876713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:32.877415 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:32.877705 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:32.878010 (+ 305us) server_negotiation.cc:299] Negotiation successful
0812 07:09:32.878435 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":328,"threads_started":1}
I20260812 07:09:32.884568 24469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e420af8532946f68cc893a598e70fd2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 32881 } }
I20260812 07:09:32.884886 24469 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:32.884958 24469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e420af8532946f68cc893a598e70fd2, State: Initialized, Role: FOLLOWER
I20260812 07:09:32.885401 24469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e420af8532946f68cc893a598e70fd2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 32881 } }
I20260812 07:09:32.885594 24469 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:32.885694 24469 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:32.885829 24469 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:32.888159 24469 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e420af8532946f68cc893a598e70fd2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 32881 } }
I20260812 07:09:32.888597 24469 leader_election.cc:302] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e420af8532946f68cc893a598e70fd2; no voters:
I20260812 07:09:32.889084 24469 leader_election.cc:288] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:32.889204 24484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:32.889853 24484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 4e420af8532946f68cc893a598e70fd2, State: Running, Role: LEADER
I20260812 07:09:32.890336 24484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e420af8532946f68cc893a598e70fd2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 32881 } }
I20260812 07:09:32.890964 24469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:32.893252 24487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e420af8532946f68cc893a598e70fd2. Latest consensus state: current_term: 1 leader_uuid: "4e420af8532946f68cc893a598e70fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e420af8532946f68cc893a598e70fd2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 32881 } } }
I20260812 07:09:32.893931 24487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:32.893236 24486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e420af8532946f68cc893a598e70fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e420af8532946f68cc893a598e70fd2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 32881 } } }
I20260812 07:09:32.894791 24486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:32.894963 24502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:32.903956 24502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:32.913251 24502 catalog_manager.cc:1347] Generated new cluster ID: b82732ade51240258df4acbf53ad75bc
I20260812 07:09:32.913369 24502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:32.941951 24502 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:32.943145 24502 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:32.956142 24502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: Generated new TSK 0
I20260812 07:09:32.956940 24502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:33.011070 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:32881
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:33.327140 24528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:33.327415 24528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:33.327574 24528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:33.334715 24528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:33.334847 24528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:33.334985 24528 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:09:33.335122 24528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:33.344868 24528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:32881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:33.346016 24528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:33.347357 24528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:33.360054 24534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:33.360018 24547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:33.360112 24540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:33.361261 24528 server_base.cc:1034] running on GCE node
I20260812 07:09:33.361893 24528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:33.362772 24528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:33.364017 24528 hybrid_clock.cc:648] HybridClock initialized: now 1786518573363954 us; error 31 us; skew 500 ppm
I20260812 07:09:33.364351 24528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:33 dist-test-slave-lgr5 krb5kdc[24316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:33.370189 24528 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:33.371785 24528 webserver.cc:460] Webserver started at http://127.21.181.65:38515/ using document root <none> and password file <none>
I20260812 07:09:33.372534 24528 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:33.372632 24528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:33.372941 24528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:33.375638 24528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "a45f955e9f874a82b2d22182de20394c"
format_stamp: "Formatted at 2026-08-12 07:09:33 on dist-test-slave-lgr5"
server_key: "06e3dd811327a1b595d36b79f9b6ccf6"
server_key_iv: "263e4da21bd059d548bebae1d30f60ba"
server_key_version: "encryptionkey@0"
I20260812 07:09:33.376430 24528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "a45f955e9f874a82b2d22182de20394c"
format_stamp: "Formatted at 2026-08-12 07:09:33 on dist-test-slave-lgr5"
server_key: "06e3dd811327a1b595d36b79f9b6ccf6"
server_key_iv: "263e4da21bd059d548bebae1d30f60ba"
server_key_version: "encryptionkey@0"
I20260812 07:09:33.381078 24528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:09:33.384236 24559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:33.385437 24528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:09:33.385582 24528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "a45f955e9f874a82b2d22182de20394c"
format_stamp: "Formatted at 2026-08-12 07:09:33 on dist-test-slave-lgr5"
server_key: "06e3dd811327a1b595d36b79f9b6ccf6"
server_key_iv: "263e4da21bd059d548bebae1d30f60ba"
server_key_version: "encryptionkey@0"
I20260812 07:09:33.385767 24528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:33.400568 24528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:33.403950 24528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:33.404206 24528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:33.406662 24528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:33.406746 24528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:33.406901 24528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:33.406963 24528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:09:33 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:33.428212 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.406977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:52063 (local address 127.21.181.126:32881)
0812 07:09:33.407462 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:33.407470 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:33.408622 (+ 1152us) server_negotiation.cc:407] Connection header received
0812 07:09:33.410584 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:33.410590 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:33.410671 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:33.410767 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:33.412766 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.413745 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.416058 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.416653 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.416767 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.421651 (+ 4884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:33.421659 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:33.421664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:33.421697 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:33.424612 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.425508 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.425515 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.425519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.425683 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.426252 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.426258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.426262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.426631 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:33.426770 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:33.427520 (+ 750us) server_negotiation.cc:299] Negotiation successful
0812 07:09:33.427761 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":221,"threads_started":1}
I20260812 07:09:33.428841 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.407579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.21.181.65:52063)
0812 07:09:33.408450 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:33.408508 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:33.410322 (+ 1814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:33.411019 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:33.411026 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:33.411700 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:33.412577 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.412610 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.413901 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.413905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.415920 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.415934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.416896 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.416905 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.417204 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.417883 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:33.417915 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:33.421408 (+ 3493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:33.424753 (+ 3345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.424762 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.424776 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.425365 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.425806 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.425815 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.425818 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.426125 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.426841 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:33.426846 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:33.427143 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 07:09:33.427347 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 07:09:33.427742 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":382,"threads_started":1}
I20260812 07:09:33.447849 24705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.434328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.21.181.65:58153)
0812 07:09:33.434848 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:33.434863 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:33.434961 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:33.435281 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:33.435284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:33.435693 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:33.435966 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.435974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.437345 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.437354 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.439123 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.439138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.439864 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.439880 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.440022 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.440614 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:33.440638 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:33.441706 (+ 1068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:33.445373 (+ 3667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.445379 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.445383 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.445894 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.446302 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.446305 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.446307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.446446 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.447011 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:33.447014 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:33.447160 (+ 146us) client_negotiation.cc:769] Sending connection context
0812 07:09:33.447273 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 07:09:33.447503 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":284,"threads_started":1}
I20260812 07:09:33.447962 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.434301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:58153 (local address 127.21.181.126:32881)
0812 07:09:33.434437 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:33.434441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:33.434945 (+ 504us) server_negotiation.cc:407] Connection header received
0812 07:09:33.435081 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:33.435087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:33.435168 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:33.435269 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:33.436128 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.437179 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.439265 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.439684 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.439791 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.441961 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:33.441967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:33.441971 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:33.442007 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:33.445174 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.446023 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.446029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.446033 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.446191 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.446534 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.446539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.446543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.446915 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:33.447060 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:33.447385 (+ 325us) server_negotiation.cc:299] Negotiation successful
0812 07:09:33.447615 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:09:33.455578 24528 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:42645
I20260812 07:09:33.455617 24772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:42645 every 8 connection(s)
I20260812 07:09:33.456723 24528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:33.458320 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24528
I20260812 07:09:33.458528 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:33.458616 24774 heartbeater.cc:347] Connected to a master server at 127.21.181.126:32881
I20260812 07:09:33.458837 24774 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:33.459153 22229 external_mini_cluster.cc:1428] Setting key 2cc9f7ab390d8b9fbff94153d39ce6dc
I20260812 07:09:33.459506 24774 heartbeater.cc:511] Master 127.21.181.126:32881 requested a full tablet report, sending...
I20260812 07:09:33.461637 24391 ts_manager.cc:194] Registered new tserver with Master: a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645)
I20260812 07:09:33.463009 24391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:52063
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:33.498459 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:32881
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:33.804816 24780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:33.805078 24780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:33.805223 24780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:33.812266 24780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:33.812371 24780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:33.812513 24780 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:09:33.812604 24780 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:33.822052 24780 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:32881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:33.823160 24780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:33.824466 24780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:33.836999 24791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:33.837062 24789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:33.836977 24796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:33.843009 24780 server_base.cc:1034] running on GCE node
I20260812 07:09:33.843544 24780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:33.844408 24780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:33.845616 24780 hybrid_clock.cc:648] HybridClock initialized: now 1786518573845553 us; error 36 us; skew 500 ppm
I20260812 07:09:33.845888 24780 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:33 dist-test-slave-lgr5 krb5kdc[24316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:33.848773 24780 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:33.850160 24780 webserver.cc:460] Webserver started at http://127.21.181.66:44283/ using document root <none> and password file <none>
I20260812 07:09:33.850852 24780 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:33.850973 24780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:33.851382 24780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:33.855094 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "d3738f9e0f3b4067b70cf2be73423fdc"
format_stamp: "Formatted at 2026-08-12 07:09:33 on dist-test-slave-lgr5"
server_key: "0011e339e71246024d8c6bba8a7c4796"
server_key_iv: "d9b6d7fea9c5799eb068c46549700c39"
server_key_version: "encryptionkey@0"
I20260812 07:09:33.856033 24780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "d3738f9e0f3b4067b70cf2be73423fdc"
format_stamp: "Formatted at 2026-08-12 07:09:33 on dist-test-slave-lgr5"
server_key: "0011e339e71246024d8c6bba8a7c4796"
server_key_iv: "d9b6d7fea9c5799eb068c46549700c39"
server_key_version: "encryptionkey@0"
I20260812 07:09:33.862094 24780 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 07:09:33.866173 24809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:33.867612 24780 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 07:09:33.867756 24780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "d3738f9e0f3b4067b70cf2be73423fdc"
format_stamp: "Formatted at 2026-08-12 07:09:33 on dist-test-slave-lgr5"
server_key: "0011e339e71246024d8c6bba8a7c4796"
server_key_iv: "d9b6d7fea9c5799eb068c46549700c39"
server_key_version: "encryptionkey@0"
I20260812 07:09:33.867965 24780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:33.889552 24780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:33.893342 24780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:33.893572 24780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:33.896202 24780 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:33.896306 24780 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:33.896445 24780 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:33.896522 24780 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:09:33 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518573, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:33.918968 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.896547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:47999 (local address 127.21.181.126:32881)
0812 07:09:33.896693 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:33.896698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:33.898103 (+ 1405us) server_negotiation.cc:407] Connection header received
0812 07:09:33.900107 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:33.900114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:33.900188 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:33.900276 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:33.902296 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.903268 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.905721 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.906233 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.906352 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.911379 (+ 5027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:33.911386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:33.911391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:33.911432 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:33.914202 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.915094 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.915122 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.915125 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.915293 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.915833 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.915838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.915841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.916222 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:33.916368 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:33.918440 (+ 2072us) server_negotiation.cc:299] Negotiation successful
0812 07:09:33.918660 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 07:09:33.919143 24824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.897197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.21.181.66:47999)
0812 07:09:33.897959 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:33.897996 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:33.899786 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:33.900524 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:33.900532 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:33.901282 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:33.902120 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.902152 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.903436 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.903442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.905570 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.905583 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.906719 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.906729 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.907021 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.907740 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:33.907781 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:33.911006 (+ 3225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:33.914332 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.914342 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.914358 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.914887 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.915408 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.915414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.915418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.915725 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.916436 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:33.916442 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:33.916912 (+ 470us) client_negotiation.cc:769] Sending connection context
0812 07:09:33.917216 (+ 304us) client_negotiation.cc:240] Negotiation successful
0812 07:09:33.917714 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":344,"threads_started":1}
I20260812 07:09:33.938009 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.924575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:37269 (local address 127.21.181.126:32881)
0812 07:09:33.924741 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:33.924746 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:33.925604 (+ 858us) server_negotiation.cc:407] Connection header received
0812 07:09:33.925713 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:33.925718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:33.925797 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:33.925899 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:33.927247 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.928201 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.929963 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.930393 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.930517 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.932229 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:33.932234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:33.932237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:33.932265 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:33.934957 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.935812 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.935817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.935821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.935943 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:33.936386 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:33.936391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:33.936394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:33.936827 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:33.936990 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:33.937466 (+ 476us) server_negotiation.cc:299] Negotiation successful
0812 07:09:33.937670 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:09:33.938112 24932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:33.924776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.21.181.66:37269)
0812 07:09:33.925507 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:33.925522 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:33.925622 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:33.925944 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:33.925949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:33.926561 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:33.927112 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.927130 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.928346 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.928355 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.929763 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:33.929783 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:33.930575 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:33.930581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:33.930733 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:33.931286 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:33.931301 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:33.932031 (+ 730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:33.935132 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.935138 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.935142 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.935702 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.936058 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:33.936063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:33.936070 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:33.936260 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:33.936968 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:33.936974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:33.937203 (+ 229us) client_negotiation.cc:769] Sending connection context
0812 07:09:33.937352 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 07:09:33.937675 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":373,"threads_started":1}
I20260812 07:09:33.946982 24780 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:36393
I20260812 07:09:33.947010 25013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:36393 every 8 connection(s)
I20260812 07:09:33.948175 24780 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:33.949942 25014 heartbeater.cc:347] Connected to a master server at 127.21.181.126:32881
I20260812 07:09:33.950131 25014 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:33.950671 25014 heartbeater.cc:511] Master 127.21.181.126:32881 requested a full tablet report, sending...
I20260812 07:09:33.951893 24391 ts_manager.cc:194] Registered new tserver with Master: d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66:36393)
I20260812 07:09:33.952061 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24780
I20260812 07:09:33.952267 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:33.952701 22229 external_mini_cluster.cc:1428] Setting key 2a3bc913cd386c2867a64190a0566dbc
I20260812 07:09:33.952756 24391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:47999
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:33.992738 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:32881
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:34.307222 25020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:34.307497 25020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:34.307632 25020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:34.314647 25020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:34.314752 25020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:34.314865 25020 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:09:34.314971 25020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:34.324293 25020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:32881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:34.325523 25020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:34.326735 25020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:34.339372 25040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:34.339450 25036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:34.339396 25035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:34.340029 25020 server_base.cc:1034] running on GCE node
I20260812 07:09:34.340586 25020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:34.341367 25020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:34.342566 25020 hybrid_clock.cc:648] HybridClock initialized: now 1786518574342504 us; error 37 us; skew 500 ppm
I20260812 07:09:34.342823 25020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:34 dist-test-slave-lgr5 krb5kdc[24316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:34.345736 25020 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:34.347098 25020 webserver.cc:460] Webserver started at http://127.21.181.67:43541/ using document root <none> and password file <none>
I20260812 07:09:34.347697 25020 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:34.347777 25020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:34.348032 25020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:34.350677 25020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "ba72b086d2724c669ea638f9d8d2040d"
format_stamp: "Formatted at 2026-08-12 07:09:34 on dist-test-slave-lgr5"
server_key: "608af103e7e03fdeedd464c0ff7017ee"
server_key_iv: "bc61a0989fab0ce20dc3e7532858f74b"
server_key_version: "encryptionkey@0"
I20260812 07:09:34.351399 25020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "ba72b086d2724c669ea638f9d8d2040d"
format_stamp: "Formatted at 2026-08-12 07:09:34 on dist-test-slave-lgr5"
server_key: "608af103e7e03fdeedd464c0ff7017ee"
server_key_iv: "bc61a0989fab0ce20dc3e7532858f74b"
server_key_version: "encryptionkey@0"
I20260812 07:09:34.356359 25020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 07:09:34.359597 25057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:34.360798 25020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:34.360940 25020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "ba72b086d2724c669ea638f9d8d2040d"
format_stamp: "Formatted at 2026-08-12 07:09:34 on dist-test-slave-lgr5"
server_key: "608af103e7e03fdeedd464c0ff7017ee"
server_key_iv: "bc61a0989fab0ce20dc3e7532858f74b"
server_key_version: "encryptionkey@0"
I20260812 07:09:34.361133 25020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:34.376344 25020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:34.380059 25020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:34.380321 25020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:34.382766 25020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:34.382848 25020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:34.382992 25020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:34.383069 25020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:09:34 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:34.405368 25073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.383663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.21.181.67:57785)
0812 07:09:34.384506 (+ 843us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.384559 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.386459 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.387141 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.387148 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.387828 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:34.388695 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.388716 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.390158 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.390163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.392816 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.392832 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.393841 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.393854 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.394159 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.394940 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:34.394977 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:34.397686 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:34.400732 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:34.400742 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:34.400757 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:34.401344 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:34.401754 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:34.401759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:34.401763 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:34.402035 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:34.402709 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:34.402716 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:34.403142 (+ 426us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.403436 (+ 294us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.403921 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":346,"threads_started":1}
I20260812 07:09:34.406374 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.383252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57785 (local address 127.21.181.126:32881)
0812 07:09:34.383456 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.383463 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.384689 (+ 1226us) server_negotiation.cc:407] Connection header received
0812 07:09:34.386726 (+ 2037us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.386732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.386813 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.386911 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:34.388942 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.389975 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.392972 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.393536 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.393667 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.397888 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:34.397893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:34.397896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:34.397928 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:34.400556 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:34.401469 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:34.401474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:34.401478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:34.401615 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:34.402192 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:34.402197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:34.402200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:34.402575 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:34.402712 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.405866 (+ 3154us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.406095 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:09:34.425395 25227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.411577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.21.181.67:34759)
0812 07:09:34.412227 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.412248 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.412385 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.412656 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.412661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.413246 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:34.413670 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.413683 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.414708 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.414714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.416488 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.416507 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.417176 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.417183 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.417371 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.418022 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:34.418044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:34.419121 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:34.422398 (+ 3277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:34.422403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:34.422410 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:34.422936 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:34.423359 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:34.423364 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:34.423368 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:34.423516 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:34.424123 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:34.424131 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:34.424410 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.424606 (+ 196us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.424858 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":313,"threads_started":1}
I20260812 07:09:34.425778 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.411580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:34759 (local address 127.21.181.126:32881)
0812 07:09:34.411740 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.411745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.412327 (+ 582us) server_negotiation.cc:407] Connection header received
0812 07:09:34.412483 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.412487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.412547 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.412617 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:34.413830 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.414599 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.416637 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.417002 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.417092 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.419277 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:34.419281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:34.419284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:34.419314 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:34.422268 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:34.423098 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:34.423104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:34.423107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:34.423264 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:34.423641 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:34.423647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:34.423650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:34.423986 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:34.424180 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.425236 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.425449 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:09:34.430205 25020 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:35425
I20260812 07:09:34.430233 25285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:35425 every 8 connection(s)
I20260812 07:09:34.431316 25020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:34.433151 25286 heartbeater.cc:347] Connected to a master server at 127.21.181.126:32881
I20260812 07:09:34.433357 25286 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:34.433926 25286 heartbeater.cc:511] Master 127.21.181.126:32881 requested a full tablet report, sending...
I20260812 07:09:34.435124 24391 ts_manager.cc:194] Registered new tserver with Master: ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425)
I20260812 07:09:34.435900 24391 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:57785
I20260812 07:09:34.440389 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25020
I20260812 07:09:34.440564 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:34.440984 22229 external_mini_cluster.cc:1428] Setting key 4aa0db29cdca15f4c7fe4eead55a3dc4
I20260812 07:09:34.443531 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:09:34.465888 24774 heartbeater.cc:503] Master 127.21.181.126:32881 was elected leader, sending a full tablet report...
Aug 12 07:09:34 dist-test-slave-lgr5 krb5kdc[24316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:09:34 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:34.490146 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.477390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35210 (local address 127.21.181.126:32881)
0812 07:09:34.477593 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.477599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.477919 (+ 320us) server_negotiation.cc:407] Connection header received
0812 07:09:34.478069 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.478074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.478149 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.478239 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:34.479761 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.480545 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.481781 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.482185 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.482294 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.485738 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:34.485743 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:34.485747 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:34.485777 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:34.487927 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:34.488565 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:34.488569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:34.488572 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:34.488698 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:34.489016 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:34.489019 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:34.489021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:34.489282 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:34.489485 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.489680 (+ 195us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.489839 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:09:34.493780 24391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:34.500391 24391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:34.522784 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.511826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36393 (local address 127.0.0.1:34708)
0812 07:09:34.513533 (+ 1707us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.513551 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.513675 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.514611 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.514620 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.514660 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.515083 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.515095 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.517076 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.517082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.519528 (+ 2446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.519546 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.521759 (+ 2213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.521766 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.522034 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.522092 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.522213 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.522388 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1514,"thread_start_us":1364,"threads_started":1}
I20260812 07:09:34.523872 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.512120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.21.181.66:36393)
0812 07:09:34.513038 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.513047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.513791 (+ 744us) server_negotiation.cc:407] Connection header received
0812 07:09:34.513905 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.513914 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.514178 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.514358 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.515288 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.516901 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.519737 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.521312 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.521448 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.523316 (+ 1868us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.523399 (+ 83us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.523583 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":1145,"threads_started":1}
I20260812 07:09:34.527650 24897 tablet_service.cc:1467] Processing CreateTablet for tablet de39e57168134b289f538a1931d51289 (DEFAULT_TABLE table=test-table [id=8ec726d03c034eb1b9681182abdfc250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:34.529042 24897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de39e57168134b289f538a1931d51289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:34.532987 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.513103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56970 (local address 127.21.181.65:42645)
0812 07:09:34.526632 (+ 13529us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.526640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.526665 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:09:34.526758 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.526765 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.526979 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.527183 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.527557 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.528971 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.531075 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.532022 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.532177 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.532368 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.532572 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.532720 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13376,"thread_start_us":13247,"threads_started":1}
I20260812 07:09:34.533208 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.511421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35425 (local address 127.0.0.1:41032)
0812 07:09:34.512215 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.512264 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.512505 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.525997 (+ 13492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.526002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.526028 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.526337 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.526349 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.528307 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.528312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.530434 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.530454 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.532403 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.532409 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.532588 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.532621 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.532689 (+ 68us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.532814 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"mutex_wait_us":3,"thread_start_us":621,"threads_started":1}
I20260812 07:09:34.532912 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.512809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:42645 (local address 127.0.0.1:56970)
0812 07:09:34.523009 (+ 10200us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.523025 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.523112 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.527195 (+ 4083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.527199 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.527226 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.527413 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.527424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.529188 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.529193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.530892 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.530903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.532193 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.532199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.532390 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.532423 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.532511 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.532643 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10124,"thread_start_us":11694,"threads_started":1}
I20260812 07:09:34.535001 25315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.511669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.21.181.67:35425)
0812 07:09:34.525104 (+ 13435us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.525111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.525144 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:09:34.525234 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.525242 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.525433 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.526019 (+ 586us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.526597 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.528144 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.530616 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.532172 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.533039 (+ 867us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.534468 (+ 1429us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.534536 (+ 68us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.534683 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13243,"thread_start_us":13436,"threads_started":1}
I20260812 07:09:34.536441 24644 tablet_service.cc:1467] Processing CreateTablet for tablet de39e57168134b289f538a1931d51289 (DEFAULT_TABLE table=test-table [id=8ec726d03c034eb1b9681182abdfc250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:34.537848 24644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de39e57168134b289f538a1931d51289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:34.538393 25166 tablet_service.cc:1467] Processing CreateTablet for tablet de39e57168134b289f538a1931d51289 (DEFAULT_TABLE table=test-table [id=8ec726d03c034eb1b9681182abdfc250]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:34.539731 25166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de39e57168134b289f538a1931d51289. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:34.541631 25327 tablet_bootstrap.cc:492] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Bootstrap starting.
I20260812 07:09:34.544591 25327 tablet_bootstrap.cc:654] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:34.545539 25327 log.cc:824] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:34.548389 25327 tablet_bootstrap.cc:492] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: No bootstrap required, opened a new log
I20260812 07:09:34.548630 25327 ts_tablet_manager.cc:1397] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:09:34.551460 25333 tablet_bootstrap.cc:492] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Bootstrap starting.
I20260812 07:09:34.553229 25335 tablet_bootstrap.cc:492] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Bootstrap starting.
I20260812 07:09:34.555079 25333 tablet_bootstrap.cc:654] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:34.556149 25335 tablet_bootstrap.cc:654] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:34.556365 25333 log.cc:824] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:34.557183 25335 log.cc:824] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:34.557864 25327 raft_consensus.cc:348] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.558292 25327 raft_consensus.cc:374] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:34.558396 25327 raft_consensus.cc:729] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3738f9e0f3b4067b70cf2be73423fdc, State: Initialized, Role: FOLLOWER
I20260812 07:09:34.559005 25327 consensus_queue.cc:260] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.559463 25333 tablet_bootstrap.cc:492] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: No bootstrap required, opened a new log
I20260812 07:09:34.559767 25333 ts_tablet_manager.cc:1397] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260812 07:09:34.565714 25014 heartbeater.cc:503] Master 127.21.181.126:32881 was elected leader, sending a full tablet report...
I20260812 07:09:34.566310 25335 tablet_bootstrap.cc:492] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: No bootstrap required, opened a new log
I20260812 07:09:34.566550 25335 ts_tablet_manager.cc:1397] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260812 07:09:34.566900 25327 ts_tablet_manager.cc:1428] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Time spent starting tablet: real 0.018s user 0.013s sys 0.004s
I20260812 07:09:34.569221 25333 raft_consensus.cc:348] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.569578 25333 raft_consensus.cc:374] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:34.569664 25333 raft_consensus.cc:729] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f955e9f874a82b2d22182de20394c, State: Initialized, Role: FOLLOWER
I20260812 07:09:34.570228 25333 consensus_queue.cc:260] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.571815 25333 ts_tablet_manager.cc:1428] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260812 07:09:34.575618 25335 raft_consensus.cc:348] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.575908 25335 raft_consensus.cc:374] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:34.576004 25335 raft_consensus.cc:729] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba72b086d2724c669ea638f9d8d2040d, State: Initialized, Role: FOLLOWER
I20260812 07:09:34.576541 25335 consensus_queue.cc:260] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.577399 25286 heartbeater.cc:503] Master 127.21.181.126:32881 was elected leader, sending a full tablet report...
I20260812 07:09:34.577817 25335 ts_tablet_manager.cc:1428] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260812 07:09:34.677443 25342 raft_consensus.cc:482] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:34.677723 25342 raft_consensus.cc:504] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.679381 25342 leader_election.cc:288] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425), a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645)
W20260812 07:09:34.682859 25289 tablet.cc:2369] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:34.683874 25343 raft_consensus.cc:482] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:34.684162 25343 raft_consensus.cc:504] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.685945 25343 leader_election.cc:288] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425), d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66:36393)
I20260812 07:09:34.686396 25315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.679818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:59221 (local address 127.21.181.67:35425)
0812 07:09:34.680060 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.680067 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.680534 (+ 467us) server_negotiation.cc:407] Connection header received
0812 07:09:34.680707 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.680711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.680786 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.680869 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.681499 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.682194 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.684519 (+ 2325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.685452 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.685607 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.685685 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.686009 (+ 324us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.686145 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:09:34.687309 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.679745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35425 (local address 127.21.181.66:59221)
0812 07:09:34.680427 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.680443 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.680579 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.681015 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.681019 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.681044 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.681357 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.681365 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.682322 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.682326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.684344 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.684360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.685664 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.685670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.685813 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.685848 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.686793 (+ 945us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.686942 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":364,"threads_started":1}
I20260812 07:09:34.687343 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.680698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:38981 (local address 127.21.181.65:42645)
0812 07:09:34.680908 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.680914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.681325 (+ 411us) server_negotiation.cc:407] Connection header received
0812 07:09:34.681506 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.681509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.681579 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.681654 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.682182 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.682916 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.684909 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.686169 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.686332 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.686409 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.686935 (+ 526us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.687100 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:09:34.688186 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.680577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:42645 (local address 127.21.181.66:38981)
0812 07:09:34.681220 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.681238 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.681375 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.681680 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.681684 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.681708 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.682037 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.682045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.683093 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.683098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.684733 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.684747 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.686559 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.686566 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.686739 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.686774 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.687545 (+ 771us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.687698 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":1,"thread_start_us":332,"threads_started":1}
I20260812 07:09:34.688609 25205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39e57168134b289f538a1931d51289" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d" is_pre_election: true
I20260812 07:09:34.689157 25205 raft_consensus.cc:2454] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 0.
I20260812 07:09:34.689775 24692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39e57168134b289f538a1931d51289" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f955e9f874a82b2d22182de20394c" is_pre_election: true
I20260812 07:09:34.690008 24815 leader_election.cc:302] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ba72b086d2724c669ea638f9d8d2040d, d3738f9e0f3b4067b70cf2be73423fdc; no voters:
I20260812 07:09:34.690325 24692 raft_consensus.cc:2454] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 0.
I20260812 07:09:34.690409 25342 raft_consensus.cc:2781] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:34.690562 25342 raft_consensus.cc:482] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:34.690676 25342 raft_consensus.cc:3037] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:34.693959 25342 raft_consensus.cc:504] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.694622 25342 leader_election.cc:288] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 election: Requested vote from peers ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425), a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645)
I20260812 07:09:34.695176 25315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.686866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:53805 (local address 127.21.181.67:35425)
0812 07:09:34.687029 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.687034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.687823 (+ 789us) server_negotiation.cc:407] Connection header received
0812 07:09:34.688106 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.688111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.688181 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.688250 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.689202 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.690280 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.692866 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.694107 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.694245 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.694334 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.694733 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.694883 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:09:34.695184 24692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39e57168134b289f538a1931d51289" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f955e9f874a82b2d22182de20394c"
I20260812 07:09:34.695392 24692 raft_consensus.cc:3037] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:34.695868 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.686326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35425 (local address 127.21.181.65:53805)
0812 07:09:34.687743 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.687760 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.687925 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.688443 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.688447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.688472 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.689025 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.689040 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.690502 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.690507 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.692672 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.692691 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.694381 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.694388 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.694527 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.694564 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.695344 (+ 780us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.695470 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1286,"mutex_wait_us":3,"thread_start_us":1246,"threads_started":1}
I20260812 07:09:34.696506 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.686985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:49575 (local address 127.21.181.66:36393)
0812 07:09:34.687165 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.687171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.689194 (+ 2023us) server_negotiation.cc:407] Connection header received
0812 07:09:34.689463 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.689469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.689542 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.689610 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:34.690697 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.691672 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.694035 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.695207 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.695385 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.695451 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.696166 (+ 715us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.696290 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:09:34.696905 25202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39e57168134b289f538a1931d51289" candidate_uuid: "a45f955e9f874a82b2d22182de20394c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d" is_pre_election: true
I20260812 07:09:34.697203 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.687228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36393 (local address 127.21.181.65:49575)
0812 07:09:34.689098 (+ 1870us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:34.689116 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:34.689296 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:34.689839 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:34.689844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:34.689924 (+ 80us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:34.690427 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.690446 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.691848 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.691851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.693789 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:34.693816 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.695711 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.695721 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:34.695925 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.695962 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:09:34.696658 (+ 696us) client_negotiation.cc:240] Negotiation successful
0812 07:09:34.696794 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1698,"mutex_wait_us":24,"thread_start_us":1369,"threads_started":1}
I20260812 07:09:34.697175 25202 raft_consensus.cc:2454] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a45f955e9f874a82b2d22182de20394c in term 0.
I20260812 07:09:34.697566 25205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39e57168134b289f538a1931d51289" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d"
I20260812 07:09:34.697763 25205 raft_consensus.cc:3037] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:34.697882 24567 leader_election.cc:302] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f955e9f874a82b2d22182de20394c, ba72b086d2724c669ea638f9d8d2040d; no voters:
I20260812 07:09:34.698545 24692 raft_consensus.cc:2454] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 1.
I20260812 07:09:34.698714 25343 raft_consensus.cc:2741] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260812 07:09:34.698622 24929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de39e57168134b289f538a1931d51289" candidate_uuid: "a45f955e9f874a82b2d22182de20394c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" is_pre_election: true
I20260812 07:09:34.698957 24815 leader_election.cc:302] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f955e9f874a82b2d22182de20394c, d3738f9e0f3b4067b70cf2be73423fdc; no voters:
I20260812 07:09:34.699203 24929 raft_consensus.cc:2379] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a45f955e9f874a82b2d22182de20394c in current term 1: Already voted for candidate d3738f9e0f3b4067b70cf2be73423fdc in this term.
I20260812 07:09:34.699505 25342 raft_consensus.cc:2781] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Leader election won for term 1
W20260812 07:09:34.699952 25017 tablet.cc:2369] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:34.700980 25205 raft_consensus.cc:2454] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 1.
I20260812 07:09:34.700990 25342 raft_consensus.cc:686] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 LEADER]: Becoming Leader. State: Replica: d3738f9e0f3b4067b70cf2be73423fdc, State: Running, Role: LEADER
I20260812 07:09:34.701462 25342 consensus_queue.cc:237] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } }
I20260812 07:09:34.706131 24391 catalog_manager.cc:5495] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc reported cstate change: term changed from 0 to 1, leader changed from <none> to d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66). New cstate: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } health_report { overall_health: UNKNOWN } } }
W20260812 07:09:34.708755 24775 tablet.cc:2369] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:34.764768 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:34.759760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.21.181.66:36393)
0812 07:09:34.759968 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:34.759981 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:34.760029 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:09:34.760340 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:34.760344 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:34.760420 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:34.760535 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:34.760817 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.761588 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.762815 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:34.763212 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:34.763300 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:34.763677 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:34.764194 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:34.764300 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:34.764423 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 07:09:34.764492 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260812 07:09:34.771817 24692 raft_consensus.cc:1261] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Refusing update from remote peer d3738f9e0f3b4067b70cf2be73423fdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:34.771901 25205 raft_consensus.cc:1261] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Refusing update from remote peer d3738f9e0f3b4067b70cf2be73423fdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:34.772763 25355 consensus_queue.cc:1035] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:34.773109 25342 consensus_queue.cc:1035] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:34.783965 25359 mvcc.cc:205] Tried to move back new op lower bound from 7317580082257092608 to 7317580081980715008. Current Snapshot: MvccSnapshot[applied={T|T < 7317580082257092608}]
I20260812 07:09:34.801162 24391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:34.801430 24391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:34.804760 24391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: Sending DeleteTablet for 3 replicas of tablet de39e57168134b289f538a1931d51289
I20260812 07:09:34.805785 24897 tablet_service.cc:1514] Processing DeleteTablet for tablet de39e57168134b289f538a1931d51289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:34 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:34708
I20260812 07:09:34.805780 25166 tablet_service.cc:1514] Processing DeleteTablet for tablet de39e57168134b289f538a1931d51289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:34 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:41032
I20260812 07:09:34.806344 24644 tablet_service.cc:1514] Processing DeleteTablet for tablet de39e57168134b289f538a1931d51289 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:34 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56970
I20260812 07:09:34.807664 24391 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:34.808393 24391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:34.818114 25165 tablet_service.cc:1467] Processing CreateTablet for tablet 49e2d774f127445fa2c3e49acaeb77d5 (DEFAULT_TABLE table=test-table [id=ab08068f90fc4bff85abcd0ad1193efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:34.818629 25371 tablet_replica.cc:330] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: stopping tablet replica
I20260812 07:09:34.818722 25165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e2d774f127445fa2c3e49acaeb77d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:34.819151 25371 raft_consensus.cc:2229] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:34.819686 25371 raft_consensus.cc:2258] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:34.820950 24640 tablet_service.cc:1467] Processing CreateTablet for tablet 49e2d774f127445fa2c3e49acaeb77d5 (DEFAULT_TABLE table=test-table [id=ab08068f90fc4bff85abcd0ad1193efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:34.821470 24640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e2d774f127445fa2c3e49acaeb77d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:34.822100 25371 ts_tablet_manager.cc:1905] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:34.824998 24896 tablet_service.cc:1467] Processing CreateTablet for tablet 49e2d774f127445fa2c3e49acaeb77d5 (DEFAULT_TABLE table=test-table [id=ab08068f90fc4bff85abcd0ad1193efe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:34.825194 25335 tablet_bootstrap.cc:492] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Bootstrap starting.
I20260812 07:09:34.825567 24896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e2d774f127445fa2c3e49acaeb77d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:34.826763 25335 tablet_bootstrap.cc:654] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:34.833705 25371 ts_tablet_manager.cc:1918] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:34.833868 25371 log.cc:1196] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/de39e57168134b289f538a1931d51289
I20260812 07:09:34.834401 25371 ts_tablet_manager.cc:1939] T de39e57168134b289f538a1931d51289 P ba72b086d2724c669ea638f9d8d2040d: Deleting consensus metadata
I20260812 07:09:34.836195 24366 catalog_manager.cc:4841] TS ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425): tablet de39e57168134b289f538a1931d51289 (table test-table [id=8ec726d03c034eb1b9681182abdfc250]) successfully deleted
I20260812 07:09:34.837817 25333 tablet_bootstrap.cc:492] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Bootstrap starting.
I20260812 07:09:34.839458 25333 tablet_bootstrap.cc:654] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:34.841316 25327 tablet_bootstrap.cc:492] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Bootstrap starting.
I20260812 07:09:34.841432 25335 tablet_bootstrap.cc:492] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: No bootstrap required, opened a new log
I20260812 07:09:34.841552 25335 ts_tablet_manager.cc:1397] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.006s
I20260812 07:09:34.842293 25335 raft_consensus.cc:348] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.842497 25335 raft_consensus.cc:374] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:34.842583 25335 raft_consensus.cc:729] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba72b086d2724c669ea638f9d8d2040d, State: Initialized, Role: FOLLOWER
I20260812 07:09:34.842964 25327 tablet_bootstrap.cc:654] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:34.842829 25335 consensus_queue.cc:260] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.844471 25335 ts_tablet_manager.cc:1428] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:09:34.845813 25374 tablet_replica.cc:330] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: stopping tablet replica
I20260812 07:09:34.846240 25374 raft_consensus.cc:2229] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:34.846726 25327 tablet_bootstrap.cc:492] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: No bootstrap required, opened a new log
I20260812 07:09:34.846921 25327 ts_tablet_manager.cc:1397] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.001s
I20260812 07:09:34.846726 25374 raft_consensus.cc:2258] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:34.847796 25372 tablet_replica.cc:330] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: stopping tablet replica
I20260812 07:09:34.848217 25372 raft_consensus.cc:2229] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:34.848340 25327 raft_consensus.cc:348] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.848663 25327 raft_consensus.cc:374] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:34.848814 25327 raft_consensus.cc:729] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3738f9e0f3b4067b70cf2be73423fdc, State: Initialized, Role: FOLLOWER
I20260812 07:09:34.848896 25372 raft_consensus.cc:2258] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:34.848924 25333 tablet_bootstrap.cc:492] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: No bootstrap required, opened a new log
I20260812 07:09:34.849059 25333 ts_tablet_manager.cc:1397] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.005s
I20260812 07:09:34.850353 25333 raft_consensus.cc:348] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.850534 25333 raft_consensus.cc:374] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:34.850636 25333 raft_consensus.cc:729] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f955e9f874a82b2d22182de20394c, State: Initialized, Role: FOLLOWER
I20260812 07:09:34.850704 25374 ts_tablet_manager.cc:1905] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:34.850816 25327 consensus_queue.cc:260] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.850905 25333 consensus_queue.cc:260] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.851512 25327 ts_tablet_manager.cc:1428] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 07:09:34.851559 25372 ts_tablet_manager.cc:1905] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:34.853626 25333 ts_tablet_manager.cc:1428] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260812 07:09:34.859117 25372 ts_tablet_manager.cc:1918] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:34.859288 25372 log.cc:1196] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/de39e57168134b289f538a1931d51289
I20260812 07:09:34.859802 25372 ts_tablet_manager.cc:1939] T de39e57168134b289f538a1931d51289 P d3738f9e0f3b4067b70cf2be73423fdc: Deleting consensus metadata
I20260812 07:09:34.861740 24364 catalog_manager.cc:4841] TS d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66:36393): tablet de39e57168134b289f538a1931d51289 (table test-table [id=8ec726d03c034eb1b9681182abdfc250]) successfully deleted
I20260812 07:09:34.864593 25374 ts_tablet_manager.cc:1918] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:09:34.864745 25374 log.cc:1196] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/de39e57168134b289f538a1931d51289
I20260812 07:09:34.865327 25374 ts_tablet_manager.cc:1939] T de39e57168134b289f538a1931d51289 P a45f955e9f874a82b2d22182de20394c: Deleting consensus metadata
I20260812 07:09:34.867162 24366 catalog_manager.cc:4841] TS a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645): tablet de39e57168134b289f538a1931d51289 (table test-table [id=8ec726d03c034eb1b9681182abdfc250]) successfully deleted
I20260812 07:09:34.976843 25355 raft_consensus.cc:482] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:34.977054 25355 raft_consensus.cc:504] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.977699 25355 leader_election.cc:288] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645), ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425)
I20260812 07:09:34.978319 24692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e2d774f127445fa2c3e49acaeb77d5" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f955e9f874a82b2d22182de20394c" is_pre_election: true
I20260812 07:09:34.978399 25205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e2d774f127445fa2c3e49acaeb77d5" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d" is_pre_election: true
I20260812 07:09:34.978560 24692 raft_consensus.cc:2454] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 0.
I20260812 07:09:34.978686 25205 raft_consensus.cc:2454] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 0.
I20260812 07:09:34.979017 24815 leader_election.cc:302] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f955e9f874a82b2d22182de20394c, d3738f9e0f3b4067b70cf2be73423fdc; no voters:
I20260812 07:09:34.979283 25355 raft_consensus.cc:2781] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:34.979457 25355 raft_consensus.cc:482] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:34.979591 25355 raft_consensus.cc:3037] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:34.981777 25355 raft_consensus.cc:504] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.982296 25355 leader_election.cc:288] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 election: Requested vote from peers a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645), ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425)
I20260812 07:09:34.982749 24692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e2d774f127445fa2c3e49acaeb77d5" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f955e9f874a82b2d22182de20394c"
I20260812 07:09:34.982829 25205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e2d774f127445fa2c3e49acaeb77d5" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d"
I20260812 07:09:34.982928 24692 raft_consensus.cc:3037] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:34.983032 25205 raft_consensus.cc:3037] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:34.984962 24692 raft_consensus.cc:2454] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 1.
I20260812 07:09:34.985311 24815 leader_election.cc:302] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f955e9f874a82b2d22182de20394c, d3738f9e0f3b4067b70cf2be73423fdc; no voters:
I20260812 07:09:34.985565 25355 raft_consensus.cc:2781] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:34.985747 25355 raft_consensus.cc:686] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 LEADER]: Becoming Leader. State: Replica: d3738f9e0f3b4067b70cf2be73423fdc, State: Running, Role: LEADER
I20260812 07:09:34.986003 25355 consensus_queue.cc:237] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:34.986151 25205 raft_consensus.cc:2454] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 1.
I20260812 07:09:34.988847 24391 catalog_manager.cc:5495] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc reported cstate change: term changed from 0 to 1, leader changed from <none> to d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66). New cstate: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } health_report { overall_health: UNKNOWN } } }
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518572, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:35.029572 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.014386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35226 (local address 127.21.181.126:32881)
0812 07:09:35.015372 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.015376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.015395 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:09:35.015436 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.015440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.015490 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.015577 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:35.017100 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.017895 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.019675 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.020070 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.020192 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.024179 (+ 3987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:35.024184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:35.024187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:35.024217 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:35.026774 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.027605 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.027609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.027611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.027732 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.028138 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.028142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.028144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.028391 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:35.028500 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.028973 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.029286 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":140,"server-negotiator.queue_time_us":882,"thread_start_us":761,"threads_started":1}
I20260812 07:09:35.029668 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.014369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:32881 (local address 127.0.0.1:35226)
0812 07:09:35.015076 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:35.015096 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:35.015242 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:35.015652 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:35.015657 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:35.016568 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:35.016931 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:35.016945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.018038 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.018044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:35.019544 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:35.019561 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.020280 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.020288 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:35.020498 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.021212 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:35.021236 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:35.023954 (+ 2718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:35.026902 (+ 2948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:35.026915 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:35.026923 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:35.027501 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:35.027834 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:35.027839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:35.027843 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:35.028019 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:35.028522 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:35.028530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:35.028726 (+ 196us) client_negotiation.cc:769] Sending connection context
0812 07:09:35.028904 (+ 178us) client_negotiation.cc:240] Negotiation successful
0812 07:09:35.029176 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":365,"threads_started":1}
I20260812 07:09:35.034952 24391 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:35226:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:09:35.036164 24391 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:35.042891 24644 tablet_service.cc:1467] Processing CreateTablet for tablet 1f454f79060f47a3ae9421f8cb92c050 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8436687f005946e3b4c22252f2d64c33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:09:35.043191 24897 tablet_service.cc:1467] Processing CreateTablet for tablet 1f454f79060f47a3ae9421f8cb92c050 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8436687f005946e3b4c22252f2d64c33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:09:35.043493 24644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f454f79060f47a3ae9421f8cb92c050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:35.043751 24897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f454f79060f47a3ae9421f8cb92c050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:35.047948 25165 tablet_service.cc:1467] Processing CreateTablet for tablet 1f454f79060f47a3ae9421f8cb92c050 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8436687f005946e3b4c22252f2d64c33]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:09:35.048532 25165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f454f79060f47a3ae9421f8cb92c050. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:35.050450 25333 tablet_bootstrap.cc:492] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: Bootstrap starting.
I20260812 07:09:35.050518 25327 tablet_bootstrap.cc:492] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: Bootstrap starting.
I20260812 07:09:35.052235 25333 tablet_bootstrap.cc:654] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:35.052393 25327 tablet_bootstrap.cc:654] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:35.056018 25335 tablet_bootstrap.cc:492] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: Bootstrap starting.
I20260812 07:09:35.057967 25335 tablet_bootstrap.cc:654] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:35.062486 25335 tablet_bootstrap.cc:492] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: No bootstrap required, opened a new log
I20260812 07:09:35.062661 25335 ts_tablet_manager.cc:1397] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:09:35.063000 25327 tablet_bootstrap.cc:492] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: No bootstrap required, opened a new log
I20260812 07:09:35.063179 25327 ts_tablet_manager.cc:1397] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260812 07:09:35.063395 25335 raft_consensus.cc:348] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.063664 25335 raft_consensus.cc:374] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:35.063756 25335 raft_consensus.cc:729] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba72b086d2724c669ea638f9d8d2040d, State: Initialized, Role: FOLLOWER
I20260812 07:09:35.064023 25335 consensus_queue.cc:260] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.064055 25327 raft_consensus.cc:348] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.064332 25327 raft_consensus.cc:374] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:35.064432 25327 raft_consensus.cc:729] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3738f9e0f3b4067b70cf2be73423fdc, State: Initialized, Role: FOLLOWER
I20260812 07:09:35.064743 25335 ts_tablet_manager.cc:1428] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:35.064622 25347 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.064744 25327 consensus_queue.cc:260] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.065025 25347 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.066139 25333 tablet_bootstrap.cc:492] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: No bootstrap required, opened a new log
I20260812 07:09:35.066303 25333 ts_tablet_manager.cc:1397] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.010s
I20260812 07:09:35.066258 25355 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.066479 25327 ts_tablet_manager.cc:1428] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260812 07:09:35.066565 25355 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.067027 25333 raft_consensus.cc:348] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.067271 25333 raft_consensus.cc:374] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:35.067364 25333 raft_consensus.cc:729] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a45f955e9f874a82b2d22182de20394c, State: Initialized, Role: FOLLOWER
I20260812 07:09:35.067623 25333 consensus_queue.cc:260] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.068105 25343 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.068384 25343 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.068877 25333 ts_tablet_manager.cc:1428] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:09:35.238593 25355 raft_consensus.cc:482] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:35.238785 25355 raft_consensus.cc:504] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.239349 25355 leader_election.cc:288] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645), ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425)
I20260812 07:09:35.239935 24692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f454f79060f47a3ae9421f8cb92c050" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f955e9f874a82b2d22182de20394c" is_pre_election: true
I20260812 07:09:35.239989 25205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f454f79060f47a3ae9421f8cb92c050" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d" is_pre_election: true
I20260812 07:09:35.240204 24692 raft_consensus.cc:2454] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 0.
I20260812 07:09:35.240283 25205 raft_consensus.cc:2454] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 0.
I20260812 07:09:35.240653 24815 leader_election.cc:302] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f955e9f874a82b2d22182de20394c, d3738f9e0f3b4067b70cf2be73423fdc; no voters:
I20260812 07:09:35.240921 25355 raft_consensus.cc:2781] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:35.241042 25355 raft_consensus.cc:482] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:35.241142 25355 raft_consensus.cc:3037] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:35.243127 25355 raft_consensus.cc:504] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.243595 25355 leader_election.cc:288] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 election: Requested vote from peers a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645), ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425)
I20260812 07:09:35.244029 24692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f454f79060f47a3ae9421f8cb92c050" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a45f955e9f874a82b2d22182de20394c"
I20260812 07:09:35.244146 25205 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f454f79060f47a3ae9421f8cb92c050" candidate_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba72b086d2724c669ea638f9d8d2040d"
I20260812 07:09:35.244272 24692 raft_consensus.cc:3037] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:35.244354 25205 raft_consensus.cc:3037] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:35.246157 24692 raft_consensus.cc:2454] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 1.
I20260812 07:09:35.246255 25205 raft_consensus.cc:2454] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3738f9e0f3b4067b70cf2be73423fdc in term 1.
I20260812 07:09:35.246554 24815 leader_election.cc:302] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a45f955e9f874a82b2d22182de20394c, d3738f9e0f3b4067b70cf2be73423fdc; no voters:
I20260812 07:09:35.246815 25355 raft_consensus.cc:2781] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:35.246973 25355 raft_consensus.cc:686] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 LEADER]: Becoming Leader. State: Replica: d3738f9e0f3b4067b70cf2be73423fdc, State: Running, Role: LEADER
I20260812 07:09:35.247429 25355 consensus_queue.cc:237] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } }
I20260812 07:09:35.248792 25365 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: TxnStatusTablet state changed. Reason: New leader d3738f9e0f3b4067b70cf2be73423fdc. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.249112 25365 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:09:35.249722 25410 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:09:35.249907 25410 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:09:35.250073 24391 catalog_manager.cc:5495] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc reported cstate change: term changed from 0 to 1, leader changed from <none> to d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66). New cstate: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } health_report { overall_health: UNKNOWN } } }
I20260812 07:09:35.305967 25395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.300546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36393 (local address 127.0.0.1:34720)
0812 07:09:35.300840 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:35.300863 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:35.300991 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:35.301441 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:35.301445 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:35.301454 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:35.301612 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:35.301619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.303003 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.303006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:35.303969 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:35.303981 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.304953 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.304957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:35.305118 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.305172 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:35.305571 (+ 399us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:35.305581 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:09:35.305634 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 07:09:35.305741 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 07:09:35.306107 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.300527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34720 (local address 127.21.181.66:36393)
0812 07:09:35.301121 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.301128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.301148 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:09:35.301224 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.301229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.301322 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.301429 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:35.301729 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.302891 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.304127 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.304824 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.304946 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.305332 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:35.305487 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:35.305575 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.305681 (+ 106us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.305761 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":326,"threads_started":1}
I20260812 07:09:35.309710 25205 raft_consensus.cc:1261] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Refusing update from remote peer d3738f9e0f3b4067b70cf2be73423fdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:35.310137 24692 raft_consensus.cc:1261] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Refusing update from remote peer d3738f9e0f3b4067b70cf2be73423fdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:35.310324 25355 consensus_queue.cc:1035] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:35.310743 25365 consensus_queue.cc:1035] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:35.313230 25347 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: TxnStatusTablet state changed. Reason: New leader d3738f9e0f3b4067b70cf2be73423fdc. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.313479 25347 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.315258 25414 mvcc.cc:205] Tried to move back new op lower bound from 7317580084462796800 to 7317580084215189504. Current Snapshot: MvccSnapshot[applied={T|T < 7317580084462796800}]
I20260812 07:09:35.321194 25355 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.321468 25355 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:09:35.325852 25347 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.326086 25347 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P ba72b086d2724c669ea638f9d8d2040d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.326972 25343 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: TxnStatusTablet state changed. Reason: New leader d3738f9e0f3b4067b70cf2be73423fdc. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.327211 25343 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.328568 25365 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.328805 25365 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P d3738f9e0f3b4067b70cf2be73423fdc: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:09:35.330199 25343 tablet_replica.cc:439] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 } } peers { permanent_uuid: "d3738f9e0f3b4067b70cf2be73423fdc" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36393 } } peers { permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 } } }
I20260812 07:09:35.330426 25343 tablet_replica.cc:442] T 1f454f79060f47a3ae9421f8cb92c050 P a45f955e9f874a82b2d22182de20394c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:35.345839 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.339903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:54553 (local address 127.21.181.66:36393)
0812 07:09:35.340104 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.340110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.340629 (+ 519us) server_negotiation.cc:407] Connection header received
0812 07:09:35.340779 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.340784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.340862 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.340951 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:35.341403 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.342511 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.343940 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.344625 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.344752 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.345182 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:35.345300 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:35.345386 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.345472 (+ 86us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.345555 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:09:35.345844 25430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.339933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36393 (local address 127.21.181.66:54553)
0812 07:09:35.340534 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:35.340549 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:35.340668 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:35.340955 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:35.340958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:35.340966 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:35.341254 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:35.341264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.342621 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.342624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:35.343803 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:35.343816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.344811 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.344817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:35.344966 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.345025 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:35.345382 (+ 357us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:35.345388 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:09:35.345438 (+ 50us) client_negotiation.cc:240] Negotiation successful
0812 07:09:35.345527 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":315,"threads_started":1}
I20260812 07:09:35.353730 24692 raft_consensus.cc:1261] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Refusing update from remote peer d3738f9e0f3b4067b70cf2be73423fdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:35.353821 25205 raft_consensus.cc:1261] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Refusing update from remote peer d3738f9e0f3b4067b70cf2be73423fdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:35.354337 25431 consensus_queue.cc:1035] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "a45f955e9f874a82b2d22182de20394c" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:35.354645 25355 consensus_queue.cc:1035] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba72b086d2724c669ea638f9d8d2040d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:35.406577 24391 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:35.406788 24391 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:35.409622 24391 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e420af8532946f68cc893a598e70fd2: Sending DeleteTablet for 3 replicas of tablet 49e2d774f127445fa2c3e49acaeb77d5
I20260812 07:09:35.410310 24644 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e2d774f127445fa2c3e49acaeb77d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:35 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56970
I20260812 07:09:35.410470 24897 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e2d774f127445fa2c3e49acaeb77d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:35 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:34708
I20260812 07:09:35.410597 25165 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e2d774f127445fa2c3e49acaeb77d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:35 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:41032
I20260812 07:09:35.420701 25440 tablet_replica.cc:330] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: stopping tablet replica
I20260812 07:09:35.420907 25440 raft_consensus.cc:2229] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:35.422554 25440 raft_consensus.cc:2258] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:35.422952 25441 tablet_replica.cc:330] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: stopping tablet replica
I20260812 07:09:35.423147 25441 raft_consensus.cc:2229] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:35.423362 25441 raft_consensus.cc:2258] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:35.424505 25440 ts_tablet_manager.cc:1905] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:35.427109 25438 tablet_replica.cc:330] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: stopping tablet replica
I20260812 07:09:35.427510 25438 raft_consensus.cc:2229] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:35.427814 25438 raft_consensus.cc:2258] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:35.428910 25438 ts_tablet_manager.cc:1905] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:35.428910 25441 ts_tablet_manager.cc:1905] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:35.437958 25440 ts_tablet_manager.cc:1918] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:35.438107 25440 log.cc:1196] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/49e2d774f127445fa2c3e49acaeb77d5
I20260812 07:09:35.438546 25440 ts_tablet_manager.cc:1939] T 49e2d774f127445fa2c3e49acaeb77d5 P d3738f9e0f3b4067b70cf2be73423fdc: Deleting consensus metadata
I20260812 07:09:35.439574 24364 catalog_manager.cc:4841] TS d3738f9e0f3b4067b70cf2be73423fdc (127.21.181.66:36393): tablet 49e2d774f127445fa2c3e49acaeb77d5 (table test-table [id=ab08068f90fc4bff85abcd0ad1193efe]) successfully deleted
I20260812 07:09:35.441694 25441 ts_tablet_manager.cc:1918] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:35.441812 25441 log.cc:1196] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/49e2d774f127445fa2c3e49acaeb77d5
I20260812 07:09:35.442265 25441 ts_tablet_manager.cc:1939] T 49e2d774f127445fa2c3e49acaeb77d5 P ba72b086d2724c669ea638f9d8d2040d: Deleting consensus metadata
I20260812 07:09:35.442386 25438 ts_tablet_manager.cc:1918] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:35.442529 25438 log.cc:1196] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/49e2d774f127445fa2c3e49acaeb77d5
I20260812 07:09:35.442981 25438 ts_tablet_manager.cc:1939] T 49e2d774f127445fa2c3e49acaeb77d5 P a45f955e9f874a82b2d22182de20394c: Deleting consensus metadata
I20260812 07:09:35.443207 24366 catalog_manager.cc:4841] TS ba72b086d2724c669ea638f9d8d2040d (127.21.181.67:35425): tablet 49e2d774f127445fa2c3e49acaeb77d5 (table test-table [id=ab08068f90fc4bff85abcd0ad1193efe]) successfully deleted
I20260812 07:09:35.444008 24366 catalog_manager.cc:4841] TS a45f955e9f874a82b2d22182de20394c (127.21.181.65:42645): tablet 49e2d774f127445fa2c3e49acaeb77d5 (table test-table [id=ab08068f90fc4bff85abcd0ad1193efe]) successfully deleted
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:35.456882 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.425823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56982 (local address 127.21.181.65:42645)
0812 07:09:35.427999 (+ 2176us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.428006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.428023 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:09:35.428075 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.428098 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.428172 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.428294 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:35.430401 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.433441 (+ 3040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.435365 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.436134 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.436244 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.448927 (+ 12683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:35.448941 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:35.448959 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:35.449004 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:35.453126 (+ 4122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.453892 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.453902 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.453906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.454069 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.454453 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.454458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.454461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.454931 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:35.455296 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.456276 (+ 980us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.456519 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2068,"thread_start_us":1917,"threads_started":1}
W20260812 07:09:35.461153 24763 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56982
I20260812 07:09:35.487577 25396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.464591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35234 (local address 127.21.181.126:32881)
0812 07:09:35.464792 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.464798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.472316 (+ 7518us) server_negotiation.cc:407] Connection header received
0812 07:09:35.472457 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.472463 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.472554 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.472644 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:35.474352 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.475472 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.477373 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.478013 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.478115 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.481077 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:35.481083 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:35.481087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:35.481126 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:35.484577 (+ 3451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.485423 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.485429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.485433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.485602 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.486009 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.486015 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.486019 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.486411 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:35.486672 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.486963 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.487192 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 07:09:35.488368 24391 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35234
I20260812 07:09:35.490257 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24528
I20260812 07:09:35.503897 25446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.491549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.21.181.65:42645)
0812 07:09:35.491913 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.491919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.492186 (+ 267us) server_negotiation.cc:407] Connection header received
0812 07:09:35.492366 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.492370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.492484 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.492577 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:35.493791 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.494508 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.495817 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.496318 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.496443 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.498657 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:35.498662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:35.498665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:35.498699 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:35.501334 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.501991 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.501996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.501998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.502125 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.502486 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.502490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.502492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.502815 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:35.503065 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.503382 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.503599 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224}
W20260812 07:09:35.504470 24763 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56994
W20260812 07:09:35.505005 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24528: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:35.505172 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24528
I20260812 07:09:35.538610 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24780
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:09:35.555629 25412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.539370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.21.181.66:36393)
0812 07:09:35.539622 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.539629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.539650 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:09:35.539782 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.539788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.539878 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.539995 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:35.541223 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.542422 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.543697 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.544468 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.544612 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.548635 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:35.548652 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:35.548672 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:35.548712 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:35.552810 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.553418 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.553427 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.553430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.553621 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.553948 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.553952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.553955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.554475 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:35.554802 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.554970 (+ 168us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.555209 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:09:35.556470 25008 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34722
W20260812 07:09:35.557112 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24780: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:35.557289 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24780
I20260812 07:09:35.587335 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25020
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[24316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518574, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:09:35.601955 25451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:35.588029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.21.181.67:35425)
0812 07:09:35.588549 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:35.588555 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:35.588575 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:09:35.588643 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:35.588647 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:35.588717 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:35.588809 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:35.590005 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.590737 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.591902 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:35.592511 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:35.592601 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:35.596540 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:35.596554 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:35.596571 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:35.596602 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:35.599368 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.599984 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.599992 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.599996 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.600164 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:35.600510 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:35.600515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:35.600519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:35.600835 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:35.601121 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:35.601412 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 07:09:35.601643 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
W20260812 07:09:35.602650 25280 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41040
W20260812 07:09:35.603200 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25020: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:35.603356 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25020
I20260812 07:09:35.627964 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24339
I20260812 07:09:35.629010 24462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:35.740324 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24339
2026-08-12T07:09:35Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4694 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[25456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[25456](info): set up 2 sockets
Aug 12 07:09:35 dist-test-slave-lgr5 krb5kdc[25456](info): commencing operation
krb5kdc: starting...
W20260812 07:09:36.962807 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:37 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:37.145484 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:42337
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:42337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:37.456149 25491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:37.456421 25491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:37.456554 25491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:37.463397 25491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:37.463495 25491 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:09:37.463565 25491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:37.463649 25491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:37.463713 25491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:37.463795 25491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:37.473440 25491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:42337
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:42337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:37.474634 25491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:37.475875 25491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:37.487032 25502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:37.487023 25500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:37.487023 25499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:37.487869 25491 server_base.cc:1034] running on GCE node
I20260812 07:09:37.488603 25491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:37.489866 25491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:37.491080 25491 hybrid_clock.cc:648] HybridClock initialized: now 1786518577491024 us; error 38 us; skew 500 ppm
I20260812 07:09:37.491384 25491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:37 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:37.494305 25491 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:37.495653 25491 webserver.cc:460] Webserver started at http://127.21.181.126:34559/ using document root <none> and password file <none>
I20260812 07:09:37.496332 25491 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:37.496433 25491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:37.496732 25491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:37.499454 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "cd4825140f544737a88cff0aad834639"
format_stamp: "Formatted at 2026-08-12 07:09:37 on dist-test-slave-lgr5"
server_key: "f887f44bc3f19e02150097ef330ace5d"
server_key_iv: "0fafcd551f3265e94b734f497c7740b9"
server_key_version: "encryptionkey@0"
I20260812 07:09:37.500265 25491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "cd4825140f544737a88cff0aad834639"
format_stamp: "Formatted at 2026-08-12 07:09:37 on dist-test-slave-lgr5"
server_key: "f887f44bc3f19e02150097ef330ace5d"
server_key_iv: "0fafcd551f3265e94b734f497c7740b9"
server_key_version: "encryptionkey@0"
I20260812 07:09:37.505067 25491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:09:37.508212 25520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:37.509361 25491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:09:37.509522 25491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "cd4825140f544737a88cff0aad834639"
format_stamp: "Formatted at 2026-08-12 07:09:37 on dist-test-slave-lgr5"
server_key: "f887f44bc3f19e02150097ef330ace5d"
server_key_iv: "0fafcd551f3265e94b734f497c7740b9"
server_key_version: "encryptionkey@0"
I20260812 07:09:37.509706 25491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:37.526787 25491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:37.530207 25491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:37.530427 25491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:37.549803 25491 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:42337
I20260812 07:09:37.549821 25630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:42337 every 8 connection(s)
I20260812 07:09:37.550956 25491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:37.551930 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25491
I20260812 07:09:37.552155 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:37.552775 22229 external_mini_cluster.cc:1428] Setting key d2adde61e9dbb4283f2abdc51920e477
I20260812 07:09:37.554736 25634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:09:37 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:37.567019 25634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Bootstrap starting.
I20260812 07:09:37.570879 25634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:37.572057 25634 log.cc:824] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:37.575119 25634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: No bootstrap required, opened a new log
I20260812 07:09:37.575224 25641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:37.554519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.21.181.126:42337)
0812 07:09:37.555240 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:37.555251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:37.555290 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:09:37.556866 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:37.556888 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:37.557238 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:37.557556 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:37.559247 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:37.560600 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:37.562253 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:37.562866 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:37.562991 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:37.567341 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:37.567351 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:37.567367 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:37.567415 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:37.571069 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:37.571710 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:37.571723 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:37.571731 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:37.571937 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:37.572270 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:37.572275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:37.572278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:37.572904 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:37.573122 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:37.573622 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 07:09:37.574004 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":322,"threads_started":1}
I20260812 07:09:37.582283 25634 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd4825140f544737a88cff0aad834639" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 42337 } }
I20260812 07:09:37.582588 25634 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:37.582659 25634 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4825140f544737a88cff0aad834639, State: Initialized, Role: FOLLOWER
I20260812 07:09:37.583117 25634 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd4825140f544737a88cff0aad834639" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 42337 } }
I20260812 07:09:37.583307 25634 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:37.583395 25634 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:37.583510 25634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:37.585518 25634 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd4825140f544737a88cff0aad834639" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 42337 } }
I20260812 07:09:37.585922 25634 leader_election.cc:302] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd4825140f544737a88cff0aad834639; no voters:
I20260812 07:09:37.586401 25634 leader_election.cc:288] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:37.586503 25645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:37.587128 25645 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [term 1 LEADER]: Becoming Leader. State: Replica: cd4825140f544737a88cff0aad834639, State: Running, Role: LEADER
I20260812 07:09:37.587636 25645 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd4825140f544737a88cff0aad834639" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 42337 } }
I20260812 07:09:37.588254 25634 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:37.590188 25648 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd4825140f544737a88cff0aad834639. Latest consensus state: current_term: 1 leader_uuid: "cd4825140f544737a88cff0aad834639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd4825140f544737a88cff0aad834639" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 42337 } } }
I20260812 07:09:37.590600 25648 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:37.591147 25646 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd4825140f544737a88cff0aad834639" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd4825140f544737a88cff0aad834639" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 42337 } } }
I20260812 07:09:37.591470 25646 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:37.592702 25660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:37.596611 25660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:37.603871 25660 catalog_manager.cc:1347] Generated new cluster ID: b406c65ac21143c9ad4fc7ce8b74ed0a
I20260812 07:09:37.603969 25660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:37.647111 25660 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:37.648370 25660 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:37.654976 25660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Generated new TSK 0
I20260812 07:09:37.655659 25660 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:37.709295 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:42337
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:38.033284 25680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:38.033667 25680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:38.033843 25680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:38.041379 25680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:38.041507 25680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:38.041656 25680 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:09:38.041729 25680 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:09:38.041846 25680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:38.051400 25680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:42337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:38.052650 25680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:38.053985 25680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:38.068332 25695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:38.068692 25694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:38.069607 25701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:38.070498 25680 server_base.cc:1034] running on GCE node
I20260812 07:09:38.071224 25680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:38.072211 25680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:38.073433 25680 hybrid_clock.cc:648] HybridClock initialized: now 1786518578073363 us; error 50 us; skew 500 ppm
I20260812 07:09:38.073751 25680 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:38 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:38.079339 25680 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:38.081139 25680 webserver.cc:460] Webserver started at http://127.21.181.65:43719/ using document root <none> and password file <none>
I20260812 07:09:38.081991 25680 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:38.082167 25680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:38.082567 25680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:38.086464 25680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "9cb012602ec44f8e9967e83057382721"
format_stamp: "Formatted at 2026-08-12 07:09:38 on dist-test-slave-lgr5"
server_key: "78ec1fe0fa58fa3e4bec208665ad7b33"
server_key_iv: "860ec57fce9c6d403fcf505c3bfe055a"
server_key_version: "encryptionkey@0"
I20260812 07:09:38.087468 25680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "9cb012602ec44f8e9967e83057382721"
format_stamp: "Formatted at 2026-08-12 07:09:38 on dist-test-slave-lgr5"
server_key: "78ec1fe0fa58fa3e4bec208665ad7b33"
server_key_iv: "860ec57fce9c6d403fcf505c3bfe055a"
server_key_version: "encryptionkey@0"
I20260812 07:09:38.094607 25680 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260812 07:09:38.098894 25713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:38.100485 25680 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:09:38.100654 25680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "9cb012602ec44f8e9967e83057382721"
format_stamp: "Formatted at 2026-08-12 07:09:38 on dist-test-slave-lgr5"
server_key: "78ec1fe0fa58fa3e4bec208665ad7b33"
server_key_iv: "860ec57fce9c6d403fcf505c3bfe055a"
server_key_version: "encryptionkey@0"
I20260812 07:09:38.100864 25680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:38.115003 25680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:38.118829 25680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:38.119076 25680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:38.121606 25680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:38.121716 25680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:38.121824 25680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:38.121901 25680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:09:38 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:38.143081 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.121938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:35997 (local address 127.21.181.126:42337)
0812 07:09:38.122451 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:38.122458 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:38.123537 (+ 1079us) server_negotiation.cc:407] Connection header received
0812 07:09:38.125567 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:38.125574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:38.125663 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:38.125791 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:38.128017 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.129165 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.131967 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.132470 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.132603 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.136890 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:38.136895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:38.136898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:38.136934 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:38.140049 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.140934 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.140938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.140940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.141064 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.141492 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.141494 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.141496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.141747 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:38.141847 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:38.142656 (+ 809us) server_negotiation.cc:299] Negotiation successful
0812 07:09:38.142820 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":244,"threads_started":1}
I20260812 07:09:38.144663 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.122510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.21.181.65:35997)
0812 07:09:38.123354 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:38.123401 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:38.125301 (+ 1900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:38.126104 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:38.126118 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:38.126865 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:38.127770 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.127801 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.129330 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.129336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.131772 (+ 2436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.131826 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.132711 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.132719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.133070 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.133766 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:38.133797 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:38.136660 (+ 2863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:38.140206 (+ 3546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.140218 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.140233 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.140822 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.141165 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.141170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.141174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.141402 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.141861 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:38.141869 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:38.142286 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 07:09:38.142587 (+ 301us) client_negotiation.cc:240] Negotiation successful
0812 07:09:38.143090 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":336,"threads_started":1}
I20260812 07:09:38.167241 25680 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:34065
I20260812 07:09:38.167315 25920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:34065 every 8 connection(s)
I20260812 07:09:38.168656 25680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:38.170795 25922 heartbeater.cc:347] Connected to a master server at 127.21.181.126:42337
I20260812 07:09:38.171025 25922 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:38.171654 25922 heartbeater.cc:511] Master 127.21.181.126:42337 requested a full tablet report, sending...
I20260812 07:09:38.172726 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.158803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:56409 (local address 127.21.181.126:42337)
0812 07:09:38.158986 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:38.158992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:38.159554 (+ 562us) server_negotiation.cc:407] Connection header received
0812 07:09:38.159727 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:38.159732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:38.159814 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:38.159908 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:38.161517 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.162236 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.164834 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.165254 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.165352 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.167339 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:38.167345 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:38.167350 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:38.167387 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:38.170022 (+ 2635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.170718 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.170723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.170726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.170876 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.171264 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.171269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.171273 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.171551 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:38.171681 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:38.172171 (+ 490us) server_negotiation.cc:299] Negotiation successful
0812 07:09:38.172384 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:09:38.172727 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25680
I20260812 07:09:38.172783 25894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.158581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.21.181.65:56409)
0812 07:09:38.159441 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:38.159459 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:38.159602 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:38.159973 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:38.159979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:38.160751 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:38.161336 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.161353 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.162371 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.162376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.164357 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.164375 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.165450 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.165457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.165634 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.166276 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:38.166296 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:38.167166 (+ 870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:38.170139 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.170143 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.170147 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.170607 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.170991 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.170997 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.171001 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.171162 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.171673 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:38.171678 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:38.171880 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 07:09:38.172007 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 07:09:38.172369 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"thread_start_us":574,"threads_started":1}
I20260812 07:09:38.172910 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:38.173688 22229 external_mini_cluster.cc:1428] Setting key 52c635cad072d01461c60aac4f875119
I20260812 07:09:38.175691 25552 ts_manager.cc:194] Registered new tserver with Master: 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065)
I20260812 07:09:38.177145 25552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:35997
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:38.215149 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:42337
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:38.526698 25928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:38.526932 25928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:38.527034 25928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:38.533835 25928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:38.533926 25928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:38.534044 25928 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:09:38.534116 25928 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:09:38.534209 25928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:38.543244 25928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:42337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:38.544330 25928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:38.545454 25928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:38.559154 25936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:38.559357 25934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:38.559725 25939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:38.560801 25928 server_base.cc:1034] running on GCE node
I20260812 07:09:38.561419 25928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:38.562460 25928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:38.563688 25928 hybrid_clock.cc:648] HybridClock initialized: now 1786518578563637 us; error 48 us; skew 500 ppm
I20260812 07:09:38.564010 25928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:38 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:38.575521 25928 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:38.576853 25928 webserver.cc:460] Webserver started at http://127.21.181.66:34369/ using document root <none> and password file <none>
I20260812 07:09:38.577464 25928 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:38.577574 25928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:38.577829 25928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:38.580615 25928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "1f369ba0672b4158b4e4fc9419f35d80"
format_stamp: "Formatted at 2026-08-12 07:09:38 on dist-test-slave-lgr5"
server_key: "57199b8a6a2309d749a3428237c73921"
server_key_iv: "b4b457add6df177bb593b0dc3996e5fb"
server_key_version: "encryptionkey@0"
I20260812 07:09:38.581295 25928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "1f369ba0672b4158b4e4fc9419f35d80"
format_stamp: "Formatted at 2026-08-12 07:09:38 on dist-test-slave-lgr5"
server_key: "57199b8a6a2309d749a3428237c73921"
server_key_iv: "b4b457add6df177bb593b0dc3996e5fb"
server_key_version: "encryptionkey@0"
I20260812 07:09:38.586038 25928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:38.589231 25950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:38.590407 25928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:38.590562 25928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1f369ba0672b4158b4e4fc9419f35d80"
format_stamp: "Formatted at 2026-08-12 07:09:38 on dist-test-slave-lgr5"
server_key: "57199b8a6a2309d749a3428237c73921"
server_key_iv: "b4b457add6df177bb593b0dc3996e5fb"
server_key_version: "encryptionkey@0"
I20260812 07:09:38.590731 25928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:38.617491 25928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:38.620955 25928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:38.621202 25928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:38.623632 25928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:38.623731 25928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:38.623839 25928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:38.623931 25928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:09:38 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518578, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:38.645030 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.624171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:45465 (local address 127.21.181.126:42337)
0812 07:09:38.624350 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:38.624356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:38.625603 (+ 1247us) server_negotiation.cc:407] Connection header received
0812 07:09:38.627730 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:38.627738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:38.627831 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:38.627946 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:38.629963 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.630913 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.633315 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.633853 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.633969 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.638363 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:38.638369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:38.638374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:38.638417 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:38.641599 (+ 3182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.642233 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.642238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.642242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.642391 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.642944 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.642950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.642954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.643314 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:38.643458 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:38.644386 (+ 928us) server_negotiation.cc:299] Negotiation successful
0812 07:09:38.644627 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:09:38.645797 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.624628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.21.181.66:45465)
0812 07:09:38.625415 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:38.625460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:38.627387 (+ 1927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:38.628309 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:38.628316 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:38.628975 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:38.629771 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.629804 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.631067 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.631072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.633177 (+ 2105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.633189 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.634090 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.634103 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.634375 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.635129 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:38.635165 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:38.638151 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:38.641729 (+ 3578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.641735 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.641745 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.642127 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.642515 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.642523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.642527 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.642820 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.643437 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:38.643444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:38.643922 (+ 478us) client_negotiation.cc:769] Sending connection context
0812 07:09:38.644262 (+ 340us) client_negotiation.cc:240] Negotiation successful
0812 07:09:38.644672 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":369,"threads_started":1}
I20260812 07:09:38.663897 26073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.650888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.21.181.66:44935)
0812 07:09:38.651527 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:38.651547 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:38.651690 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:38.652182 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:38.652185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:38.652806 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:38.653250 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.653265 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.654319 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.654324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.655459 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:38.655473 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.656181 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.656188 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:38.656357 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.656969 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:38.656985 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:38.658011 (+ 1026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:38.661277 (+ 3266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.661283 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.661290 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.661691 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.662041 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:38.662046 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:38.662050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:38.662170 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:38.662848 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:38.662852 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:38.663057 (+ 205us) client_negotiation.cc:769] Sending connection context
0812 07:09:38.663224 (+ 167us) client_negotiation.cc:240] Negotiation successful
0812 07:09:38.663488 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":332,"threads_started":1}
I20260812 07:09:38.663937 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:38.650873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:44935 (local address 127.21.181.126:42337)
0812 07:09:38.651065 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:38.651072 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:38.651640 (+ 568us) server_negotiation.cc:407] Connection header received
0812 07:09:38.651844 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:38.651850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:38.651922 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:38.652008 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:38.653385 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.654191 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.655593 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:38.655977 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:38.656120 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:38.658229 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:38.658233 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:38.658236 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:38.658264 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:38.661174 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.661800 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.661806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.661809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.661933 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:38.662272 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:38.662277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:38.662281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:38.662654 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:38.662793 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:38.663372 (+ 579us) server_negotiation.cc:299] Negotiation successful
0812 07:09:38.663605 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:09:38.673147 25928 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:39811
I20260812 07:09:38.673168 26157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:39811 every 8 connection(s)
I20260812 07:09:38.674273 25928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:38.676128 26158 heartbeater.cc:347] Connected to a master server at 127.21.181.126:42337
I20260812 07:09:38.676312 26158 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:38.676956 26158 heartbeater.cc:511] Master 127.21.181.126:42337 requested a full tablet report, sending...
I20260812 07:09:38.678119 25552 ts_manager.cc:194] Registered new tserver with Master: 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811)
I20260812 07:09:38.678889 25552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:45465
I20260812 07:09:38.681181 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25928
I20260812 07:09:38.681314 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:38.681695 22229 external_mini_cluster.cc:1428] Setting key 7d33b1a0400923fd638968a81ded130b
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:38.722576 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:42337
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:39.028723 26163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:39.028965 26163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:39.029067 26163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:39.036361 26163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:39.036451 26163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:39.036543 26163 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260812 07:09:39.036595 26163 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:09:39.036659 26163 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:39.045675 26163 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:42337
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:39.046754 26163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:39.048058 26163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:39.060034 26168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:39.060048 26171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:39.060365 26174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:39.061038 26163 server_base.cc:1034] running on GCE node
I20260812 07:09:39.061574 26163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:39.062386 26163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:39.063581 26163 hybrid_clock.cc:648] HybridClock initialized: now 1786518579063531 us; error 52 us; skew 500 ppm
I20260812 07:09:39.063866 26163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:39 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:39.066767 26163 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:39.068121 26163 webserver.cc:460] Webserver started at http://127.21.181.67:45861/ using document root <none> and password file <none>
I20260812 07:09:39.068797 26163 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:39.068928 26163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:39.069358 26163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:39.072074 26163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "e6719daae3be4f439b73e12070c10089"
format_stamp: "Formatted at 2026-08-12 07:09:39 on dist-test-slave-lgr5"
server_key: "cf6ae5c426ae52e3bb1e64a88cc376cf"
server_key_iv: "0906aea1a9922ada91c9d33a9522a99d"
server_key_version: "encryptionkey@0"
I20260812 07:09:39.072846 26163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "e6719daae3be4f439b73e12070c10089"
format_stamp: "Formatted at 2026-08-12 07:09:39 on dist-test-slave-lgr5"
server_key: "cf6ae5c426ae52e3bb1e64a88cc376cf"
server_key_iv: "0906aea1a9922ada91c9d33a9522a99d"
server_key_version: "encryptionkey@0"
I20260812 07:09:39.077673 26163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:39.080822 26189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:39.082051 26163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:09:39.082187 26163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "e6719daae3be4f439b73e12070c10089"
format_stamp: "Formatted at 2026-08-12 07:09:39 on dist-test-slave-lgr5"
server_key: "cf6ae5c426ae52e3bb1e64a88cc376cf"
server_key_iv: "0906aea1a9922ada91c9d33a9522a99d"
server_key_version: "encryptionkey@0"
I20260812 07:09:39.082352 26163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:39.105798 26163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:39.109405 26163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:39.109699 26163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:39.112602 26163 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:39.112707 26163 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:39.112818 26163 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:39.112898 26163 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:09:39 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518579, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:39.134807 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.113176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:37261 (local address 127.21.181.126:42337)
0812 07:09:39.113364 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.113370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.114704 (+ 1334us) server_negotiation.cc:407] Connection header received
0812 07:09:39.116853 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.116860 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.116949 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.117041 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:39.119061 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.120249 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.122577 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.123123 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.123239 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.127308 (+ 4069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:39.127315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:39.127320 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:39.127361 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:39.130607 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.131579 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.131585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.131588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.131768 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.132558 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.132563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.132567 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.132942 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:39.133105 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.134130 (+ 1025us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.134392 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:09:39.136418 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.113655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.21.181.67:37261)
0812 07:09:39.114531 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.114579 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.116594 (+ 2015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.117325 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.117336 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.118067 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:39.118812 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.118847 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.120415 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.120421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.122428 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.122441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.123364 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.123377 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.123625 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.124448 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:39.124496 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:39.127080 (+ 2584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:39.130773 (+ 3693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:39.130781 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:39.130796 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:39.131432 (+ 636us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:39.131919 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:39.131924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:39.131927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:39.132386 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:39.133128 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:39.133142 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:39.133570 (+ 428us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.133941 (+ 371us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.134496 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":381,"threads_started":1}
I20260812 07:09:39.151794 26163 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:45115
I20260812 07:09:39.151875 26394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:45115 every 8 connection(s)
I20260812 07:09:39.153072 26163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:39.157202 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26163
I20260812 07:09:39.157388 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:39.157881 22229 external_mini_cluster.cc:1428] Setting key e540cfee0c8478c991344e82a6e95ce5
I20260812 07:09:39.158304 26351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.143648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.21.181.67:38871)
0812 07:09:39.144284 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.144308 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.144454 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.144952 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.144957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.145610 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:39.146054 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.146071 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.147477 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.147483 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.149206 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.149224 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.150187 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.150194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.150373 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.151002 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:39.151033 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:39.151947 (+ 914us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:39.155687 (+ 3740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:39.155692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:39.155696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:39.156250 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:39.156679 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:39.156684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:39.156688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:39.156848 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:39.157508 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:39.157513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:39.157664 (+ 151us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.157771 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.158010 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":435,"threads_started":1}
I20260812 07:09:39.158464 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.144014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:38871 (local address 127.21.181.126:42337)
0812 07:09:39.144226 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.144232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.144386 (+ 154us) server_negotiation.cc:407] Connection header received
0812 07:09:39.144573 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.144579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.144655 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.144766 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:39.146319 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.147228 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.149390 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.149990 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.150122 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.152208 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:39.152212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:39.152216 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:39.152247 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:39.155495 (+ 3248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.156380 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.156384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.156386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.156543 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.156970 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.156983 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.156987 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.157368 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:39.157524 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.157919 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.158149 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 07:09:39.158859 26398 heartbeater.cc:347] Connected to a master server at 127.21.181.126:42337
I20260812 07:09:39.159091 26398 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:39.159921 26398 heartbeater.cc:511] Master 127.21.181.126:42337 requested a full tablet report, sending...
I20260812 07:09:39.161808 25550 ts_manager.cc:194] Registered new tserver with Master: e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:39.162940 25550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:37261
I20260812 07:09:39.173818 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:09:39.179661 25922 heartbeater.cc:503] Master 127.21.181.126:42337 was elected leader, sending a full tablet report...
Aug 12 07:09:39 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:09:39 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:39.221645 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.209238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.21.181.126:42337)
0812 07:09:39.209457 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.209464 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.209701 (+ 237us) server_negotiation.cc:407] Connection header received
0812 07:09:39.209835 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.209839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.209912 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.209992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:39.211267 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.211960 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.213162 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.213586 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.213676 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.217137 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:39.217141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:39.217144 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:39.217175 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:39.219557 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.220049 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.220052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.220054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.220203 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.220499 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.220501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.220503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.220745 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:39.220927 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.221140 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.221298 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260812 07:09:39.225250 25550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:39.231911 25550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:39.259624 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.243492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.21.181.65:34065)
0812 07:09:39.244278 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.244287 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.245746 (+ 1459us) server_negotiation.cc:407] Connection header received
0812 07:09:39.247321 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.247331 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.247607 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.247820 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.248871 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.251660 (+ 2789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.255677 (+ 4017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.257293 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.257410 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.257661 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.259143 (+ 1482us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.259320 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":803,"threads_started":1}
I20260812 07:09:39.260638 26423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.243211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.0.0.1:60996)
0812 07:09:39.245572 (+ 2361us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.245626 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.247051 (+ 1425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.248160 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.248168 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.248210 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.248646 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.248660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.251886 (+ 3226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.251892 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.255479 (+ 3587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.255497 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.257854 (+ 2357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.257861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.258133 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.258199 (+ 66us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.260049 (+ 1850us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.260285 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2089,"thread_start_us":15292,"threads_started":1}
I20260812 07:09:39.264678 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 4411e2c8e9694fc5809fcf4abead72a7 (DEFAULT_TABLE table=test-table [id=8cfff01ac89e4c3db1becb77eebe6b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:39.265898 26420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.243498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:45115 (local address 127.0.0.1:58372)
0812 07:09:39.258283 (+ 14785us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.258308 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.258422 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.259710 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.259715 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.259743 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.260157 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.260168 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.261847 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.261851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.263329 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.263343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.265190 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.265197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.265356 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.265389 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.265469 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.265590 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14660,"thread_start_us":1655,"threads_started":1}
I20260812 07:09:39.266173 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4411e2c8e9694fc5809fcf4abead72a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.266717 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.244124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58372 (local address 127.21.181.67:45115)
0812 07:09:39.259140 (+ 15016us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.259147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.259179 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:09:39.259270 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.259277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.259514 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.259802 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.260380 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.261695 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.263485 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.264693 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.264812 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.265046 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.266253 (+ 1207us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.266408 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14855,"thread_start_us":14671,"threads_started":1}
I20260812 07:09:39.268205 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.246576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.21.181.66:39811)
0812 07:09:39.260514 (+ 13938us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.260522 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.261288 (+ 766us) server_negotiation.cc:407] Connection header received
0812 07:09:39.261404 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.261411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.261613 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.261809 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.262428 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.263841 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.266195 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.267290 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.267394 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.267539 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.267828 (+ 289us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.267940 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13784,"thread_start_us":13643,"threads_started":1}
I20260812 07:09:39.268240 26426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.246068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39811 (local address 127.0.0.1:58118)
0812 07:09:39.261029 (+ 14961us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.261049 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.261171 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.261805 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.261809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.261835 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.262237 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.262248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.263990 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.263995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.266014 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.266029 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.267495 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.267501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.267671 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.267703 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.267797 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.267914 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14835,"thread_start_us":14605,"threads_started":1}
I20260812 07:09:39.271296 26274 tablet_service.cc:1467] Processing CreateTablet for tablet 4411e2c8e9694fc5809fcf4abead72a7 (DEFAULT_TABLE table=test-table [id=8cfff01ac89e4c3db1becb77eebe6b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:39.271778 26046 tablet_service.cc:1467] Processing CreateTablet for tablet 4411e2c8e9694fc5809fcf4abead72a7 (DEFAULT_TABLE table=test-table [id=8cfff01ac89e4c3db1becb77eebe6b5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:39.272814 26274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4411e2c8e9694fc5809fcf4abead72a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.273074 26046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4411e2c8e9694fc5809fcf4abead72a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.277529 26430 tablet_bootstrap.cc:492] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Bootstrap starting.
I20260812 07:09:39.280685 26430 tablet_bootstrap.cc:654] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.282003 26430 log.cc:824] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:39.284373 26430 tablet_bootstrap.cc:492] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: No bootstrap required, opened a new log
I20260812 07:09:39.284619 26430 ts_tablet_manager.cc:1397] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:09:39.287145 26432 tablet_bootstrap.cc:492] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Bootstrap starting.
I20260812 07:09:39.288518 26433 tablet_bootstrap.cc:492] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Bootstrap starting.
I20260812 07:09:39.290724 26432 tablet_bootstrap.cc:654] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.291448 26433 tablet_bootstrap.cc:654] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.292028 26432 log.cc:824] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:39.292629 26433 log.cc:824] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:39.294119 26430 raft_consensus.cc:348] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.294488 26430 raft_consensus.cc:374] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.294587 26430 raft_consensus.cc:729] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.294983 26432 tablet_bootstrap.cc:492] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: No bootstrap required, opened a new log
I20260812 07:09:39.295269 26432 ts_tablet_manager.cc:1397] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:09:39.295212 26430 consensus_queue.cc:260] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.295696 26433 tablet_bootstrap.cc:492] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: No bootstrap required, opened a new log
I20260812 07:09:39.295965 26433 ts_tablet_manager.cc:1397] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:09:39.296926 26430 ts_tablet_manager.cc:1428] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260812 07:09:39.305392 26432 raft_consensus.cc:348] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.305828 26432 raft_consensus.cc:374] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.305939 26432 raft_consensus.cc:729] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6719daae3be4f439b73e12070c10089, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.305908 26433 raft_consensus.cc:348] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.306303 26433 raft_consensus.cc:374] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.306404 26433 raft_consensus.cc:729] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.306592 26432 consensus_queue.cc:260] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.307235 26433 consensus_queue.cc:260] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.307605 26438 raft_consensus.cc:482] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:39.307708 26398 heartbeater.cc:503] Master 127.21.181.126:42337 was elected leader, sending a full tablet report...
I20260812 07:09:39.307956 26438 raft_consensus.cc:504] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.308387 26432 ts_tablet_manager.cc:1428] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260812 07:09:39.316318 26438 leader_election.cc:288] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:39.317157 26158 heartbeater.cc:503] Master 127.21.181.126:42337 was elected leader, sending a full tablet report...
I20260812 07:09:39.320387 26433 ts_tablet_manager.cc:1428] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Time spent starting tablet: real 0.024s user 0.013s sys 0.012s
I20260812 07:09:39.325402 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.310411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:44839 (local address 127.21.181.67:45115)
0812 07:09:39.310594 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.310600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.319366 (+ 8766us) server_negotiation.cc:407] Connection header received
0812 07:09:39.319435 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.319441 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.319520 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.319602 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.320430 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.321560 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.323689 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.324508 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.324659 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.324733 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.325048 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.325171 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:09:39.325465 26444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.310075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:45115 (local address 127.21.181.66:44839)
0812 07:09:39.318771 (+ 8696us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.318792 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.318935 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.319775 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.319779 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.319806 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.320257 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.320272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.321691 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.321696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.323542 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.323555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.324726 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.324736 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.324896 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.324921 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.325010 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.325139 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8547,"thread_start_us":8334,"threads_started":1}
I20260812 07:09:39.326352 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4411e2c8e9694fc5809fcf4abead72a7" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089" is_pre_election: true
I20260812 07:09:39.326858 26298 raft_consensus.cc:2454] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 0.
I20260812 07:09:39.327016 26446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.310209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.21.181.66:33129)
0812 07:09:39.321613 (+ 11404us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.321637 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.321769 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.322254 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.322259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.322284 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.322743 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.322757 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.323710 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.323714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.325048 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.325062 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.326400 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.326404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.326526 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.326549 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.326632 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.326749 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11285,"mutex_wait_us":1,"thread_start_us":11157,"threads_started":1}
I20260812 07:09:39.327040 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.310580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:33129 (local address 127.21.181.65:34065)
0812 07:09:39.310755 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.310761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.321717 (+ 10956us) server_negotiation.cc:407] Connection header received
0812 07:09:39.321907 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.321912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.322007 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.322092 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.322902 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.323577 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.325201 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.326216 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.326382 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.326451 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.326672 (+ 221us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.326798 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:09:39.327538 25951 leader_election.cc:302] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, e6719daae3be4f439b73e12070c10089; no voters:
I20260812 07:09:39.327905 26443 raft_consensus.cc:2781] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:39.327813 25842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4411e2c8e9694fc5809fcf4abead72a7" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb012602ec44f8e9967e83057382721" is_pre_election: true
I20260812 07:09:39.328052 26443 raft_consensus.cc:482] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:39.328224 26443 raft_consensus.cc:3037] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:39.328356 25842 raft_consensus.cc:2454] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 0.
I20260812 07:09:39.330441 26443 raft_consensus.cc:504] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.330971 26443 leader_election.cc:288] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:39.331341 25842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4411e2c8e9694fc5809fcf4abead72a7" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb012602ec44f8e9967e83057382721"
I20260812 07:09:39.331384 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4411e2c8e9694fc5809fcf4abead72a7" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089"
I20260812 07:09:39.331545 25842 raft_consensus.cc:3037] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:39.331549 26298 raft_consensus.cc:3037] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:39.334584 26298 raft_consensus.cc:2454] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 1.
I20260812 07:09:39.334584 25842 raft_consensus.cc:2454] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 1.
I20260812 07:09:39.334977 25951 leader_election.cc:302] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, 9cb012602ec44f8e9967e83057382721, e6719daae3be4f439b73e12070c10089; no voters:
I20260812 07:09:39.335254 26443 raft_consensus.cc:2781] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:39.335407 26443 raft_consensus.cc:686] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 LEADER]: Becoming Leader. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Running, Role: LEADER
I20260812 07:09:39.335852 26443 consensus_queue.cc:237] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } }
I20260812 07:09:39.340341 25550 catalog_manager.cc:5495] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "1f369ba0672b4158b4e4fc9419f35d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } health_report { overall_health: HEALTHY } } }
Aug 12 07:09:39 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518577, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:39.402335 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.390275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:42337 (local address 127.0.0.1:33222)
0812 07:09:39.390813 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.390831 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.390989 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.391286 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.391289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.391947 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:39.392277 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.392287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.393206 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.393210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.394125 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.394136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.394863 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.394868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.395001 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.395615 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:39.395637 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:39.397738 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:39.400239 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:39.400249 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:39.400255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:39.400632 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:39.400958 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:39.400961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:39.400964 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:39.401120 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:39.401561 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:39.401567 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:39.401708 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.401832 (+ 124us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.402031 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
I20260812 07:09:39.402401 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.390217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.21.181.126:42337)
0812 07:09:39.390362 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.390366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.390921 (+ 555us) server_negotiation.cc:407] Connection header received
0812 07:09:39.391088 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.391092 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.391162 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.391239 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:39.392403 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.393103 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.394250 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.394715 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.394802 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.397892 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:39.397897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:39.397900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:39.397929 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:39.400119 (+ 2190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.400732 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.400735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.400738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.400872 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:39.401206 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:39.401209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:39.401211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:39.401458 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:39.401564 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.401920 (+ 356us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.402152 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":48}
W20260812 07:09:39.406541 26399 tablet.cc:2369] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:39.406836 25550 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:33222:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:09:39.407907 25550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:39.414048 25804 tablet_service.cc:1467] Processing CreateTablet for tablet e6d18eb4bc16490bb279e558c5f9764d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e29236b26a3147bdbda0e4645306446d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:09:39.414534 26274 tablet_service.cc:1467] Processing CreateTablet for tablet e6d18eb4bc16490bb279e558c5f9764d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e29236b26a3147bdbda0e4645306446d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:09:39.415078 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d18eb4bc16490bb279e558c5f9764d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.414732 26046 tablet_service.cc:1467] Processing CreateTablet for tablet e6d18eb4bc16490bb279e558c5f9764d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=e29236b26a3147bdbda0e4645306446d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:09:39.415300 26274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d18eb4bc16490bb279e558c5f9764d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.415572 26046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6d18eb4bc16490bb279e558c5f9764d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.420125 26432 tablet_bootstrap.cc:492] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: Bootstrap starting.
I20260812 07:09:39.421392 26432 tablet_bootstrap.cc:654] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.421932 26430 tablet_bootstrap.cc:492] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: Bootstrap starting.
W20260812 07:09:39.421998 25923 tablet.cc:2369] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:39.422241 26433 tablet_bootstrap.cc:492] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: Bootstrap starting.
I20260812 07:09:39.423535 26430 tablet_bootstrap.cc:654] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.423863 26433 tablet_bootstrap.cc:654] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.424855 26432 tablet_bootstrap.cc:492] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: No bootstrap required, opened a new log
I20260812 07:09:39.425007 26432 ts_tablet_manager.cc:1397] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 07:09:39.425794 26430 tablet_bootstrap.cc:492] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: No bootstrap required, opened a new log
I20260812 07:09:39.425927 26430 ts_tablet_manager.cc:1397] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:09:39.425761 26432 raft_consensus.cc:348] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.426038 26432 raft_consensus.cc:374] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260812 07:09:39.426052 26159 tablet.cc:2369] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:09:39.426131 26432 raft_consensus.cc:729] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6719daae3be4f439b73e12070c10089, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.426584 26430 raft_consensus.cc:348] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.426810 26430 raft_consensus.cc:374] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.426726 26432 consensus_queue.cc:260] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.426883 26430 raft_consensus.cc:729] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.426939 26433 tablet_bootstrap.cc:492] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: No bootstrap required, opened a new log
I20260812 07:09:39.427091 26433 ts_tablet_manager.cc:1397] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260812 07:09:39.427058 26430 consensus_queue.cc:260] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.427230 26440 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.427603 26440 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.427631 26430 ts_tablet_manager.cc:1428] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:39.427811 26433 raft_consensus.cc:348] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.428041 26433 raft_consensus.cc:374] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.428160 26433 raft_consensus.cc:729] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.427949 26437 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.428272 26437 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.428380 26433 consensus_queue.cc:260] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.428951 26433 ts_tablet_manager.cc:1428] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:39.428851 26443 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.429142 26443 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.429319 26432 ts_tablet_manager.cc:1428] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260812 07:09:39.646140 26437 raft_consensus.cc:482] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:39.646375 26437 raft_consensus.cc:504] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.647889 26437 leader_election.cc:288] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:39.654841 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.648231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:35785 (local address 127.21.181.66:39811)
0812 07:09:39.648430 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.648436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.648826 (+ 390us) server_negotiation.cc:407] Connection header received
0812 07:09:39.649033 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.649041 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.649129 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.649223 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.649921 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.650912 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.653000 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.653911 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.654040 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.654095 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.654453 (+ 358us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.654590 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:09:39.654883 26465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.648160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39811 (local address 127.21.181.65:35785)
0812 07:09:39.648712 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.648727 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.648873 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.649286 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.649290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.649319 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.649764 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.649777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.651064 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.651069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.652828 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.652844 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.654128 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.654135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.654293 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.654329 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.654416 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.654553 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":34,"thread_start_us":288,"threads_started":1}
I20260812 07:09:39.655408 26467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.648863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:45115 (local address 127.21.181.65:54841)
0812 07:09:39.649558 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.649574 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.649705 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.650009 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.650012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.650032 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:39.650286 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.650292 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.651557 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.651562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.653261 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.653277 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.654506 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.654512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.654684 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.654720 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.654817 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.655100 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"mutex_wait_us":4,"thread_start_us":344,"threads_started":1}
I20260812 07:09:39.655416 26425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.649008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:54841 (local address 127.21.181.67:45115)
0812 07:09:39.649183 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.649189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.649664 (+ 475us) server_negotiation.cc:407] Connection header received
0812 07:09:39.649821 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.649824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.649893 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.649998 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:39.650428 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.651417 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.653414 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.654262 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.654407 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.654482 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.654865 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.655177 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:09:39.655884 26081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d18eb4bc16490bb279e558c5f9764d" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f369ba0672b4158b4e4fc9419f35d80" is_pre_election: true
I20260812 07:09:39.656283 26081 raft_consensus.cc:2454] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 0.
I20260812 07:09:39.656704 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d18eb4bc16490bb279e558c5f9764d" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089" is_pre_election: true
I20260812 07:09:39.656864 25716 leader_election.cc:302] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, 9cb012602ec44f8e9967e83057382721; no voters:
I20260812 07:09:39.656989 26298 raft_consensus.cc:2454] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 0.
I20260812 07:09:39.658070 26437 raft_consensus.cc:2781] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:39.658185 26437 raft_consensus.cc:482] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:39.658279 26437 raft_consensus.cc:3037] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:39.660362 26437 raft_consensus.cc:504] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.660876 26437 leader_election.cc:288] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 election: Requested vote from peers 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:39.661317 26081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d18eb4bc16490bb279e558c5f9764d" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f369ba0672b4158b4e4fc9419f35d80"
I20260812 07:09:39.661351 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6d18eb4bc16490bb279e558c5f9764d" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089"
I20260812 07:09:39.661526 26298 raft_consensus.cc:3037] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:39.661530 26081 raft_consensus.cc:3037] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:39.664400 26298 raft_consensus.cc:2454] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 1.
I20260812 07:09:39.664422 26081 raft_consensus.cc:2454] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 1.
I20260812 07:09:39.664762 25716 leader_election.cc:302] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb012602ec44f8e9967e83057382721, e6719daae3be4f439b73e12070c10089; no voters:
I20260812 07:09:39.665004 26437 raft_consensus.cc:2781] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:39.665421 26437 raft_consensus.cc:686] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Running, Role: LEADER
I20260812 07:09:39.666061 26437 consensus_queue.cc:237] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.669147 26468 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: TxnStatusTablet state changed. Reason: New leader 9cb012602ec44f8e9967e83057382721. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.669456 26468 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:09:39.670159 26477 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:09:39.670423 26477 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:09:39.670883 25546 catalog_manager.cc:5495] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb012602ec44f8e9967e83057382721 (127.21.181.65). New cstate: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } health_report { overall_health: UNKNOWN } } }
I20260812 07:09:39.722867 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.717023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51454 (local address 127.21.181.65:34065)
0812 07:09:39.717224 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.717231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.717331 (+ 100us) server_negotiation.cc:407] Connection header received
0812 07:09:39.717562 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.717568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.717675 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.717838 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:39.718121 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.719314 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.720584 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.721220 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.721357 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.721729 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.722206 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.722341 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.722450 (+ 109us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.722534 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 07:09:39.722859 26459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.717022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.0.0.1:51454)
0812 07:09:39.717246 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.717265 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.717382 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.717835 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.717838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.717848 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:39.718007 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.718015 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.719417 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.719420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.720444 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.720455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.721347 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.721351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.721487 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.721550 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.722314 (+ 764us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.722337 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.722416 (+ 79us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.722569 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:09:39.730170 26081 raft_consensus.cc:1261] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Refusing update from remote peer 9cb012602ec44f8e9967e83057382721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:39.730271 26298 raft_consensus.cc:1261] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Refusing update from remote peer 9cb012602ec44f8e9967e83057382721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:39.731005 26468 consensus_queue.cc:1035] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:39.731305 26437 consensus_queue.cc:1035] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:39.740654 26440 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: TxnStatusTablet state changed. Reason: New leader 9cb012602ec44f8e9967e83057382721. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.740911 26440 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.742318 26482 mvcc.cc:205] Tried to move back new op lower bound from 7317580102565933056 to 7317580102315687936. Current Snapshot: MvccSnapshot[applied={T|T < 7317580102565933056}]
I20260812 07:09:39.743935 26443 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: TxnStatusTablet state changed. Reason: New leader 9cb012602ec44f8e9967e83057382721. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.744249 26443 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.754828 26440 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.755110 26440 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P e6719daae3be4f439b73e12070c10089: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.760658 26443 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.760878 26443 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 1f369ba0672b4158b4e4fc9419f35d80: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:09:39.765635 26437 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.765924 26437 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:09:39.767050 26468 tablet_replica.cc:439] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } } }
I20260812 07:09:39.767310 26468 tablet_replica.cc:442] T e6d18eb4bc16490bb279e558c5f9764d P 9cb012602ec44f8e9967e83057382721: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:09:39.776461 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.770775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.21.181.66:39811)
0812 07:09:39.770938 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.770945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.771379 (+ 434us) server_negotiation.cc:407] Connection header received
0812 07:09:39.771584 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.771589 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.771670 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.771804 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:39.772339 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.773098 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.774413 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.774822 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.774927 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.775339 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.775859 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.775986 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.776119 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.776199 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:09:39.794095 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.785868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:57559 (local address 127.21.181.65:34065)
0812 07:09:39.786311 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.786317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.788717 (+ 2400us) server_negotiation.cc:407] Connection header received
0812 07:09:39.788907 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.788911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.788974 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.789047 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:39.789583 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.790278 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.792296 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.792717 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.792806 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.793332 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.793429 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.793517 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.793691 (+ 174us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.793771 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301}
I20260812 07:09:39.794265 26495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.785735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.21.181.66:57559)
0812 07:09:39.788607 (+ 2872us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.788627 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.788764 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.789080 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.789085 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.789096 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:39.789421 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.789433 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.790397 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.790403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.792169 (+ 1766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.792185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.792878 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.792884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.793079 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.793146 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.793533 (+ 387us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.793546 (+ 13us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.793655 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.793791 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2707,"thread_start_us":2522,"threads_started":1}
I20260812 07:09:39.803550 26298 raft_consensus.cc:1261] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Refusing update from remote peer 1f369ba0672b4158b4e4fc9419f35d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:39.803773 25842 raft_consensus.cc:1261] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Refusing update from remote peer 1f369ba0672b4158b4e4fc9419f35d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:39.804235 26443 consensus_queue.cc:1035] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:39.804539 26438 consensus_queue.cc:1035] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:39.841745 26429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.834697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:45459 (local address 127.21.181.66:39811)
0812 07:09:39.834860 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:39.834866 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:39.835369 (+ 503us) server_negotiation.cc:407] Connection header received
0812 07:09:39.835568 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:39.835572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:39.835629 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:39.835696 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:39.836562 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.837649 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.839804 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.840367 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.840501 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.841088 (+ 587us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.841192 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.841265 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:39.841420 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 07:09:39.841495 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:09:39.842046 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:39.834642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39811 (local address 127.21.181.65:45459)
0812 07:09:39.835262 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:39.835307 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:39.835461 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:39.835893 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:39.835898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:39.835912 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:39.836349 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.836363 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.837836 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.837841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.839672 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:39.839689 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:39.840656 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:39.840663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:39.840826 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:39.840918 (+ 92us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:39.841294 (+ 376us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:39.841313 (+ 19us) client_negotiation.cc:769] Sending connection context
0812 07:09:39.841386 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:09:39.841541 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":278,"threads_started":1}
I20260812 07:09:39.877934 25550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:39.878229 25550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:39.882025 25550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Sending DeleteTablet for 3 replicas of tablet 4411e2c8e9694fc5809fcf4abead72a7
I20260812 07:09:39.883522 26046 tablet_service.cc:1514] Processing DeleteTablet for tablet 4411e2c8e9694fc5809fcf4abead72a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:39 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58118
I20260812 07:09:39.883747 26274 tablet_service.cc:1514] Processing DeleteTablet for tablet 4411e2c8e9694fc5809fcf4abead72a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:39 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 07:09:39.884554 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 4411e2c8e9694fc5809fcf4abead72a7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:39 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:60996
I20260812 07:09:39.887926 25550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:39.889843 25550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:39.900292 25803 tablet_service.cc:1467] Processing CreateTablet for tablet 6826b2408a9d40f18e0526126699a4bf (DEFAULT_TABLE table=test-table [id=0c58c97955ec4d03b761c1c58c37a792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:39.900964 25803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6826b2408a9d40f18e0526126699a4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.901271 26514 tablet_replica.cc:330] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: stopping tablet replica
I20260812 07:09:39.902452 26514 raft_consensus.cc:2229] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:39.903122 26514 raft_consensus.cc:2258] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:39.907536 26514 ts_tablet_manager.cc:1905] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:39.908162 26045 tablet_service.cc:1467] Processing CreateTablet for tablet 6826b2408a9d40f18e0526126699a4bf (DEFAULT_TABLE table=test-table [id=0c58c97955ec4d03b761c1c58c37a792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:39.908718 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6826b2408a9d40f18e0526126699a4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.914110 26520 tablet_replica.cc:330] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: stopping tablet replica
I20260812 07:09:39.914645 26520 raft_consensus.cc:2229] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:39.915020 26520 raft_consensus.cc:2258] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:39.915702 26433 tablet_bootstrap.cc:492] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Bootstrap starting.
I20260812 07:09:39.917626 26433 tablet_bootstrap.cc:654] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.918640 26514 ts_tablet_manager.cc:1918] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:39.918807 26514 log.cc:1196] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/4411e2c8e9694fc5809fcf4abead72a7
I20260812 07:09:39.919337 26514 ts_tablet_manager.cc:1939] T 4411e2c8e9694fc5809fcf4abead72a7 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting consensus metadata
I20260812 07:09:39.921034 25522 catalog_manager.cc:4841] TS 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811): tablet 4411e2c8e9694fc5809fcf4abead72a7 (table test-table [id=8cfff01ac89e4c3db1becb77eebe6b5b]) successfully deleted
I20260812 07:09:39.915892 26430 tablet_bootstrap.cc:492] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Bootstrap starting.
I20260812 07:09:39.925868 26433 tablet_bootstrap.cc:492] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: No bootstrap required, opened a new log
I20260812 07:09:39.926115 26433 ts_tablet_manager.cc:1397] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260812 07:09:39.926617 26430 tablet_bootstrap.cc:654] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.927000 26433 raft_consensus.cc:348] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.927697 26433 raft_consensus.cc:374] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.927853 26433 raft_consensus.cc:729] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.928232 26433 consensus_queue.cc:260] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.931434 26433 ts_tablet_manager.cc:1428] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 07:09:39.932528 26518 tablet_replica.cc:330] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: stopping tablet replica
I20260812 07:09:39.932915 26272 tablet_service.cc:1467] Processing CreateTablet for tablet 6826b2408a9d40f18e0526126699a4bf (DEFAULT_TABLE table=test-table [id=0c58c97955ec4d03b761c1c58c37a792]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:39.933038 26518 raft_consensus.cc:2229] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:39.933444 26272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6826b2408a9d40f18e0526126699a4bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:39.934089 26520 ts_tablet_manager.cc:1905] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:39.937115 26430 tablet_bootstrap.cc:492] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: No bootstrap required, opened a new log
I20260812 07:09:39.937317 26430 ts_tablet_manager.cc:1397] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.001s
I20260812 07:09:39.938139 26430 raft_consensus.cc:348] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.933454 26518 raft_consensus.cc:2258] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:39.938441 26430 raft_consensus.cc:374] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.938593 26430 raft_consensus.cc:729] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.939286 26430 consensus_queue.cc:260] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.940068 26430 ts_tablet_manager.cc:1428] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260812 07:09:39.941061 26518 ts_tablet_manager.cc:1905] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:39.945900 26520 ts_tablet_manager.cc:1918] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:39.946058 26520 log.cc:1196] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/4411e2c8e9694fc5809fcf4abead72a7
I20260812 07:09:39.946576 26520 ts_tablet_manager.cc:1939] T 4411e2c8e9694fc5809fcf4abead72a7 P 9cb012602ec44f8e9967e83057382721: Deleting consensus metadata
I20260812 07:09:39.947993 26524 tablet_bootstrap.cc:492] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Bootstrap starting.
I20260812 07:09:39.948403 25524 catalog_manager.cc:4841] TS 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065): tablet 4411e2c8e9694fc5809fcf4abead72a7 (table test-table [id=8cfff01ac89e4c3db1becb77eebe6b5b]) successfully deleted
I20260812 07:09:39.949853 26524 tablet_bootstrap.cc:654] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:39.952718 26524 tablet_bootstrap.cc:492] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: No bootstrap required, opened a new log
I20260812 07:09:39.952858 26524 ts_tablet_manager.cc:1397] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:09:39.953608 26524 raft_consensus.cc:348] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.953862 26524 raft_consensus.cc:374] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:39.953792 26518 ts_tablet_manager.cc:1918] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:39.954123 26518 log.cc:1196] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/4411e2c8e9694fc5809fcf4abead72a7
I20260812 07:09:39.954677 26518 ts_tablet_manager.cc:1939] T 4411e2c8e9694fc5809fcf4abead72a7 P e6719daae3be4f439b73e12070c10089: Deleting consensus metadata
I20260812 07:09:39.953951 26524 raft_consensus.cc:729] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6719daae3be4f439b73e12070c10089, State: Initialized, Role: FOLLOWER
I20260812 07:09:39.955204 26524 consensus_queue.cc:260] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:39.956598 25522 catalog_manager.cc:4841] TS e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115): tablet 4411e2c8e9694fc5809fcf4abead72a7 (table test-table [id=8cfff01ac89e4c3db1becb77eebe6b5b]) successfully deleted
I20260812 07:09:39.957623 26524 ts_tablet_manager.cc:1428] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260812 07:09:40.121449 26468 raft_consensus.cc:482] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:40.121714 26468 raft_consensus.cc:504] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:40.122567 26468 leader_election.cc:288] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:40.123292 26081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826b2408a9d40f18e0526126699a4bf" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f369ba0672b4158b4e4fc9419f35d80" is_pre_election: true
I20260812 07:09:40.123291 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826b2408a9d40f18e0526126699a4bf" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089" is_pre_election: true
I20260812 07:09:40.123579 26298 raft_consensus.cc:2454] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 0.
I20260812 07:09:40.123594 26081 raft_consensus.cc:2454] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 0.
I20260812 07:09:40.124131 25716 leader_election.cc:302] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb012602ec44f8e9967e83057382721, e6719daae3be4f439b73e12070c10089; no voters:
I20260812 07:09:40.124480 26468 raft_consensus.cc:2781] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:40.124599 26468 raft_consensus.cc:482] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:40.124708 26468 raft_consensus.cc:3037] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:40.127967 26468 raft_consensus.cc:504] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:40.128648 26468 leader_election.cc:288] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 election: Requested vote from peers 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:40.129204 26081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826b2408a9d40f18e0526126699a4bf" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f369ba0672b4158b4e4fc9419f35d80"
I20260812 07:09:40.129218 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6826b2408a9d40f18e0526126699a4bf" candidate_uuid: "9cb012602ec44f8e9967e83057382721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089"
I20260812 07:09:40.129419 26081 raft_consensus.cc:3037] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:40.129438 26298 raft_consensus.cc:3037] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:40.132349 26298 raft_consensus.cc:2454] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 1.
I20260812 07:09:40.132407 26081 raft_consensus.cc:2454] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9cb012602ec44f8e9967e83057382721 in term 1.
I20260812 07:09:40.132880 25716 leader_election.cc:302] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, 9cb012602ec44f8e9967e83057382721; no voters:
I20260812 07:09:40.133162 26468 raft_consensus.cc:2781] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:40.133388 26468 raft_consensus.cc:686] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 1 LEADER]: Becoming Leader. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Running, Role: LEADER
I20260812 07:09:40.133721 26468 consensus_queue.cc:237] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:40.136669 25550 catalog_manager.cc:5495] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cb012602ec44f8e9967e83057382721 (127.21.181.65). New cstate: current_term: 1 leader_uuid: "9cb012602ec44f8e9967e83057382721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } health_report { overall_health: UNKNOWN } } }
I20260812 07:09:40.215663 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:40.210853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51464 (local address 127.21.181.65:34065)
0812 07:09:40.211026 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:40.211032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:40.211088 (+ 56us) server_negotiation.cc:407] Connection header received
0812 07:09:40.211260 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:40.211265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:40.211353 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:40.211456 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:40.211773 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:40.212636 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:40.214034 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:40.214479 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:40.214598 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:40.214995 (+ 397us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:40.215119 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:40.215196 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:40.215349 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 07:09:40.215429 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:09:40.228816 26507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:40.223890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.21.181.65:45819)
0812 07:09:40.224071 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:40.224108 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:40.224205 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:40.224506 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:40.224509 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:40.224518 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:40.224740 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:40.224748 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:40.226014 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:40.226017 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:40.226941 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:40.226953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:40.227813 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:40.227817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:40.227953 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:40.227988 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:40.228400 (+ 412us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:40.228410 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:09:40.228469 (+ 59us) client_negotiation.cc:240] Negotiation successful
0812 07:09:40.228569 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":17}
I20260812 07:09:40.228889 26424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:40.223919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:45819 (local address 127.21.181.65:34065)
0812 07:09:40.224112 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:40.224118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:40.224155 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:09:40.224310 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:40.224315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:40.224396 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:40.224500 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:40.224884 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:40.225897 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:40.227082 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:40.227676 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:40.227805 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:40.228165 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:40.228306 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:40.228402 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:40.228494 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 07:09:40.228570 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":61}
I20260812 07:09:40.236816 26298 raft_consensus.cc:1261] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Refusing update from remote peer 9cb012602ec44f8e9967e83057382721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:40.236903 26081 raft_consensus.cc:1261] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Refusing update from remote peer 9cb012602ec44f8e9967e83057382721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:40.237422 26437 consensus_queue.cc:1035] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:40.237746 26468 consensus_queue.cc:1035] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:40.242798 26481 mvcc.cc:205] Tried to move back new op lower bound from 7317580104645476352 to 7317580104230268928. Current Snapshot: MvccSnapshot[applied={T|T < 7317580104645476352}]
I20260812 07:09:40.247215 26483 mvcc.cc:205] Tried to move back new op lower bound from 7317580104645476352 to 7317580104230268928. Current Snapshot: MvccSnapshot[applied={T|T < 7317580104645476352}]
W20260812 07:09:40.265583 26481 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 07:09:41.854372 25546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:41.854612 25546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:41.858373 25546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Sending DeleteTablet for 3 replicas of tablet 6826b2408a9d40f18e0526126699a4bf
I20260812 07:09:41.859067 26045 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826b2408a9d40f18e0526126699a4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:41 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58118
I20260812 07:09:41.859097 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826b2408a9d40f18e0526126699a4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:41 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:60996
I20260812 07:09:41.859151 26272 tablet_service.cc:1514] Processing DeleteTablet for tablet 6826b2408a9d40f18e0526126699a4bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:41 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 07:09:41.871811 26566 tablet_replica.cc:330] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: stopping tablet replica
I20260812 07:09:41.872038 26566 raft_consensus.cc:2229] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:41.872315 26566 raft_consensus.cc:2258] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:41.872824 26565 tablet_replica.cc:330] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: stopping tablet replica
I20260812 07:09:41.873049 26564 tablet_replica.cc:330] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: stopping tablet replica
I20260812 07:09:41.873023 26565 raft_consensus.cc:2229] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:41.873229 26564 raft_consensus.cc:2229] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:41.873250 26565 raft_consensus.cc:2258] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:41.873766 26564 raft_consensus.cc:2258] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:41.874090 26566 ts_tablet_manager.cc:1905] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:41.874600 26565 ts_tablet_manager.cc:1905] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:41.874720 26564 ts_tablet_manager.cc:1905] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:41.880721 26566 ts_tablet_manager.cc:1918] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:09:41.880771 26564 ts_tablet_manager.cc:1918] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:09:41.880865 26566 log.cc:1196] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/6826b2408a9d40f18e0526126699a4bf
I20260812 07:09:41.880851 26565 ts_tablet_manager.cc:1918] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:09:41.880895 26564 log.cc:1196] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/6826b2408a9d40f18e0526126699a4bf
I20260812 07:09:41.880992 26565 log.cc:1196] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/6826b2408a9d40f18e0526126699a4bf
I20260812 07:09:41.881316 26564 ts_tablet_manager.cc:1939] T 6826b2408a9d40f18e0526126699a4bf P 9cb012602ec44f8e9967e83057382721: Deleting consensus metadata
I20260812 07:09:41.881318 26566 ts_tablet_manager.cc:1939] T 6826b2408a9d40f18e0526126699a4bf P e6719daae3be4f439b73e12070c10089: Deleting consensus metadata
I20260812 07:09:41.881403 26565 ts_tablet_manager.cc:1939] T 6826b2408a9d40f18e0526126699a4bf P 1f369ba0672b4158b4e4fc9419f35d80: Deleting consensus metadata
I20260812 07:09:41.882422 25522 catalog_manager.cc:4841] TS e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115): tablet 6826b2408a9d40f18e0526126699a4bf (table test-table [id=0c58c97955ec4d03b761c1c58c37a792]) successfully deleted
I20260812 07:09:41.882422 25524 catalog_manager.cc:4841] TS 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065): tablet 6826b2408a9d40f18e0526126699a4bf (table test-table [id=0c58c97955ec4d03b761c1c58c37a792]) successfully deleted
I20260812 07:09:41.882735 25522 catalog_manager.cc:4841] TS 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811): tablet 6826b2408a9d40f18e0526126699a4bf (table test-table [id=0c58c97955ec4d03b761c1c58c37a792]) successfully deleted
Aug 12 07:09:41 dist-test-slave-lgr5 krb5kdc[25456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:09:41 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:41.917567 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:41.904434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.21.181.126:42337)
0812 07:09:41.905010 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:41.905015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:41.905093 (+ 78us) server_negotiation.cc:407] Connection header received
0812 07:09:41.905259 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:41.905263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:41.905322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:41.905409 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:41.906732 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.907511 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.908771 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.909179 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.909271 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:41.912826 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:41.912833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:41.912838 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:41.912872 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:41.915380 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:41.916001 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:41.916005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:41.916007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:41.916176 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:41.916472 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:41.916476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:41.916479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:41.916726 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:41.916905 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:41.917082 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:09:41.917243 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":283,"threads_started":1}
I20260812 07:09:41.920590 25546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:41.921192 25546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:41.927152 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 012018c97d184d758cb8239fc82027f2 (DEFAULT_TABLE table=test-table [id=90bdc54f0f264e478041c58830eae6fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:41.927366 26045 tablet_service.cc:1467] Processing CreateTablet for tablet 012018c97d184d758cb8239fc82027f2 (DEFAULT_TABLE table=test-table [id=90bdc54f0f264e478041c58830eae6fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:41.927568 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 012018c97d184d758cb8239fc82027f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:41.927802 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 012018c97d184d758cb8239fc82027f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:41.927769 26272 tablet_service.cc:1467] Processing CreateTablet for tablet 012018c97d184d758cb8239fc82027f2 (DEFAULT_TABLE table=test-table [id=90bdc54f0f264e478041c58830eae6fd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:41.928205 26272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 012018c97d184d758cb8239fc82027f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:41.932765 26585 tablet_bootstrap.cc:492] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Bootstrap starting.
I20260812 07:09:41.934551 26586 tablet_bootstrap.cc:492] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Bootstrap starting.
I20260812 07:09:41.934376 26585 tablet_bootstrap.cc:654] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:41.935036 26589 tablet_bootstrap.cc:492] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Bootstrap starting.
I20260812 07:09:41.936340 26586 tablet_bootstrap.cc:654] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:41.936635 26589 tablet_bootstrap.cc:654] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:41.938623 26586 tablet_bootstrap.cc:492] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: No bootstrap required, opened a new log
I20260812 07:09:41.938768 26586 ts_tablet_manager.cc:1397] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260812 07:09:41.939599 26586 raft_consensus.cc:348] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.939846 26586 raft_consensus.cc:374] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:41.939937 26586 raft_consensus.cc:729] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6719daae3be4f439b73e12070c10089, State: Initialized, Role: FOLLOWER
I20260812 07:09:41.940248 26586 consensus_queue.cc:260] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.941082 26586 ts_tablet_manager.cc:1428] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260812 07:09:41.943080 26585 tablet_bootstrap.cc:492] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: No bootstrap required, opened a new log
I20260812 07:09:41.943229 26585 ts_tablet_manager.cc:1397] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260812 07:09:41.944139 26589 tablet_bootstrap.cc:492] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: No bootstrap required, opened a new log
I20260812 07:09:41.943970 26585 raft_consensus.cc:348] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.944236 26585 raft_consensus.cc:374] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:41.944278 26589 ts_tablet_manager.cc:1397] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260812 07:09:41.944353 26585 raft_consensus.cc:729] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Initialized, Role: FOLLOWER
I20260812 07:09:41.944600 26585 consensus_queue.cc:260] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.945009 26589 raft_consensus.cc:348] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.945202 26585 ts_tablet_manager.cc:1428] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:41.945269 26589 raft_consensus.cc:374] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:41.945361 26589 raft_consensus.cc:729] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Initialized, Role: FOLLOWER
I20260812 07:09:41.945595 26589 consensus_queue.cc:260] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.946462 26589 ts_tablet_manager.cc:1428] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:09:41.975523 26595 raft_consensus.cc:482] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:41.975735 26595 raft_consensus.cc:504] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.977469 26595 leader_election.cc:288] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065), 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811)
I20260812 07:09:41.989872 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:41.978160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57639 (local address 127.21.181.65:34065)
0812 07:09:41.984476 (+ 6316us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:41.984481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:41.984500 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:09:41.984554 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:41.984557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:41.984611 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:41.984696 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:41.985261 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.986210 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.988054 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.988932 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.989110 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:41.989186 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:41.989423 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 07:09:41.989548 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6212,"thread_start_us":6063,"threads_started":1}
I20260812 07:09:41.989928 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:41.977657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.21.181.67:57639)
0812 07:09:41.978211 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:41.978226 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:41.978336 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:41.984745 (+ 6409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:41.984749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:41.984777 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:41.985060 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:41.985070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.986375 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.986379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:41.987913 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:41.987927 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.989102 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.989108 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:41.989276 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:41.989301 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:09:41.989384 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:09:41.989522 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"spinlock_wait_cycles":16256,"thread_start_us":317,"threads_started":1}
I20260812 07:09:41.990124 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:41.977757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39811 (local address 127.21.181.67:56921)
0812 07:09:41.984709 (+ 6952us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:41.984724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:41.984848 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:41.985165 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:41.985169 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:41.985192 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:09:41.985535 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:41.985544 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.986659 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.986665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:41.988148 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:41.988164 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.989336 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.989342 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:41.989509 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:41.989533 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:09:41.989614 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 07:09:41.989730 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6836,"thread_start_us":6699,"threads_started":1}
I20260812 07:09:41.990363 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:41.978083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:56921 (local address 127.21.181.66:39811)
0812 07:09:41.978736 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:41.978740 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:41.984806 (+ 6066us) server_negotiation.cc:407] Connection header received
0812 07:09:41.984981 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:41.984987 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:41.985053 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:41.985133 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:09:41.985748 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.986546 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.988292 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:41.989137 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:41.989292 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:41.989363 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:41.989653 (+ 290us) server_negotiation.cc:299] Negotiation successful
0812 07:09:41.989774 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":349,"threads_started":1}
I20260812 07:09:41.990545 25842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012018c97d184d758cb8239fc82027f2" candidate_uuid: "e6719daae3be4f439b73e12070c10089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb012602ec44f8e9967e83057382721" is_pre_election: true
I20260812 07:09:41.990798 25842 raft_consensus.cc:2454] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6719daae3be4f439b73e12070c10089 in term 0.
I20260812 07:09:41.991020 26078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012018c97d184d758cb8239fc82027f2" candidate_uuid: "e6719daae3be4f439b73e12070c10089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f369ba0672b4158b4e4fc9419f35d80" is_pre_election: true
I20260812 07:09:41.991254 26078 raft_consensus.cc:2454] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6719daae3be4f439b73e12070c10089 in term 0.
I20260812 07:09:41.991315 26194 leader_election.cc:302] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9cb012602ec44f8e9967e83057382721, e6719daae3be4f439b73e12070c10089; no voters:
I20260812 07:09:41.991720 26595 raft_consensus.cc:2781] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:41.991858 26595 raft_consensus.cc:482] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:41.991961 26595 raft_consensus.cc:3037] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:41.994467 26595 raft_consensus.cc:504] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:41.995041 26595 leader_election.cc:288] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065), 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811)
I20260812 07:09:41.995433 25842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012018c97d184d758cb8239fc82027f2" candidate_uuid: "e6719daae3be4f439b73e12070c10089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb012602ec44f8e9967e83057382721"
I20260812 07:09:41.995482 26078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "012018c97d184d758cb8239fc82027f2" candidate_uuid: "e6719daae3be4f439b73e12070c10089" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f369ba0672b4158b4e4fc9419f35d80"
I20260812 07:09:41.995608 25842 raft_consensus.cc:3037] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:41.995649 26078 raft_consensus.cc:3037] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:41.997642 26078 raft_consensus.cc:2454] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6719daae3be4f439b73e12070c10089 in term 1.
I20260812 07:09:41.997722 25842 raft_consensus.cc:2454] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6719daae3be4f439b73e12070c10089 in term 1.
I20260812 07:09:41.998080 26193 leader_election.cc:302] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, 9cb012602ec44f8e9967e83057382721, e6719daae3be4f439b73e12070c10089; no voters:
I20260812 07:09:41.998395 26595 raft_consensus.cc:2781] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:41.998795 26595 raft_consensus.cc:686] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 1 LEADER]: Becoming Leader. State: Replica: e6719daae3be4f439b73e12070c10089, State: Running, Role: LEADER
I20260812 07:09:41.999126 26595 consensus_queue.cc:237] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.002256 25550 catalog_manager.cc:5495] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6719daae3be4f439b73e12070c10089 (127.21.181.67). New cstate: current_term: 1 leader_uuid: "e6719daae3be4f439b73e12070c10089" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } health_report { overall_health: HEALTHY } } }
I20260812 07:09:42.025028 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:42.020065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.21.181.67:45115)
0812 07:09:42.020556 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:42.020560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:42.020579 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:09:42.020652 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:42.020656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:42.020724 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:42.020856 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:42.021159 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.021894 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.023085 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.023491 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.023578 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:42.023937 (+ 359us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:42.024517 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:42.024628 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:42.024709 (+ 81us) server_negotiation.cc:299] Negotiation successful
0812 07:09:42.024771 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":250,"threads_started":1}
I20260812 07:09:42.038138 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:42.033618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:53315 (local address 127.21.181.65:34065)
0812 07:09:42.033762 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:42.033766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:42.034163 (+ 397us) server_negotiation.cc:407] Connection header received
0812 07:09:42.034368 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:42.034371 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:42.034429 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:42.034496 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:42.034902 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.035574 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.036835 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.037232 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.037319 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:42.037659 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:42.037744 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:42.037802 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:42.037890 (+ 88us) server_negotiation.cc:299] Negotiation successful
0812 07:09:42.037947 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:09:42.038254 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:42.033488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34065 (local address 127.21.181.67:53315)
0812 07:09:42.034086 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:42.034105 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:42.034279 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:42.034517 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:42.034520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:42.034529 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:42.034791 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:42.034798 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.035669 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.035672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:42.036732 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:42.036744 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.037360 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.037365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:42.037489 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:42.037535 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:42.037808 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:42.037815 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:09:42.037868 (+ 53us) client_negotiation.cc:240] Negotiation successful
0812 07:09:42.037957 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":305,"threads_started":1}
I20260812 07:09:42.045176 26078 raft_consensus.cc:1261] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Refusing update from remote peer e6719daae3be4f439b73e12070c10089: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:42.045249 25842 raft_consensus.cc:1261] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Refusing update from remote peer e6719daae3be4f439b73e12070c10089: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:42.045751 26604 consensus_queue.cc:1035] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:42.046067 26595 consensus_queue.cc:1035] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:42.070225 26608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:42.064948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:52391 (local address 127.21.181.67:45115)
0812 07:09:42.065079 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:42.065083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:42.065485 (+ 402us) server_negotiation.cc:407] Connection header received
0812 07:09:42.065654 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:42.065657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:42.065710 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:42.065784 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:42.066430 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.067217 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.068665 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.069097 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.069224 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:42.069602 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:42.069695 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:42.069767 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:42.069900 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 07:09:42.069973 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:09:42.070470 26619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:42.064870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:45115 (local address 127.21.181.65:52391)
0812 07:09:42.065409 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:42.065424 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:42.065537 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:42.065957 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:42.065961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:42.065971 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:09:42.066238 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:42.066251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.067348 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.067353 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:42.068536 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:42.068549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.069292 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.069297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:42.069421 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:42.069457 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:09:42.069794 (+ 337us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:09:42.069804 (+ 10us) client_negotiation.cc:769] Sending connection context
0812 07:09:42.069871 (+ 67us) client_negotiation.cc:240] Negotiation successful
0812 07:09:42.070019 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":293,"threads_started":1}
I20260812 07:09:42.101106 25550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:42.101383 25550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:42.104394 25550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Sending DeleteTablet for 3 replicas of tablet 012018c97d184d758cb8239fc82027f2
I20260812 07:09:42.105264 26045 tablet_service.cc:1514] Processing DeleteTablet for tablet 012018c97d184d758cb8239fc82027f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:42 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58118
I20260812 07:09:42.105276 26272 tablet_service.cc:1514] Processing DeleteTablet for tablet 012018c97d184d758cb8239fc82027f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:42 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 07:09:42.105532 26565 tablet_replica.cc:330] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: stopping tablet replica
I20260812 07:09:42.105552 26566 tablet_replica.cc:330] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: stopping tablet replica
I20260812 07:09:42.105749 26566 raft_consensus.cc:2229] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:42.105777 26565 raft_consensus.cc:2229] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:42.105969 26565 raft_consensus.cc:2258] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:42.106241 26566 raft_consensus.cc:2258] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:42.106191 25550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:09:42.106793 25550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:09:42.107100 26565 ts_tablet_manager.cc:1905] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:42.107736 26566 ts_tablet_manager.cc:1905] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:42.111171 25804 tablet_service.cc:1514] Processing DeleteTablet for tablet 012018c97d184d758cb8239fc82027f2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:42 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:60996
I20260812 07:09:42.111440 26564 tablet_replica.cc:330] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: stopping tablet replica
I20260812 07:09:42.111629 26564 raft_consensus.cc:2229] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:42.111790 26564 raft_consensus.cc:2258] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:42.112717 26564 ts_tablet_manager.cc:1905] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:42.113381 26565 ts_tablet_manager.cc:1918] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:42.113545 26565 log.cc:1196] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/012018c97d184d758cb8239fc82027f2
I20260812 07:09:42.113778 26045 tablet_service.cc:1467] Processing CreateTablet for tablet 847d79be5e70489b997db63ed8121e30 (DEFAULT_TABLE table=test-table [id=317b74fda3214626a81589c59d54c573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:42.113999 26565 ts_tablet_manager.cc:1939] T 012018c97d184d758cb8239fc82027f2 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting consensus metadata
I20260812 07:09:42.114189 26045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847d79be5e70489b997db63ed8121e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:42.116367 25522 catalog_manager.cc:4841] TS 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811): tablet 012018c97d184d758cb8239fc82027f2 (table test-table [id=90bdc54f0f264e478041c58830eae6fd]) successfully deleted
I20260812 07:09:42.117165 25804 tablet_service.cc:1467] Processing CreateTablet for tablet 847d79be5e70489b997db63ed8121e30 (DEFAULT_TABLE table=test-table [id=317b74fda3214626a81589c59d54c573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:42.117560 25804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847d79be5e70489b997db63ed8121e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:42.119490 26589 tablet_bootstrap.cc:492] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Bootstrap starting.
I20260812 07:09:42.120249 26564 ts_tablet_manager.cc:1918] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:42.120397 26564 log.cc:1196] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/012018c97d184d758cb8239fc82027f2
I20260812 07:09:42.120879 26564 ts_tablet_manager.cc:1939] T 012018c97d184d758cb8239fc82027f2 P 9cb012602ec44f8e9967e83057382721: Deleting consensus metadata
I20260812 07:09:42.121054 26589 tablet_bootstrap.cc:654] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:42.122516 26272 tablet_service.cc:1467] Processing CreateTablet for tablet 847d79be5e70489b997db63ed8121e30 (DEFAULT_TABLE table=test-table [id=317b74fda3214626a81589c59d54c573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:09:42.122903 26272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 847d79be5e70489b997db63ed8121e30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:42.123693 26585 tablet_bootstrap.cc:492] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Bootstrap starting.
I20260812 07:09:42.125236 26585 tablet_bootstrap.cc:654] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:42.127163 26585 tablet_bootstrap.cc:492] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: No bootstrap required, opened a new log
I20260812 07:09:42.127300 26585 ts_tablet_manager.cc:1397] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260812 07:09:42.128005 26585 raft_consensus.cc:348] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.128247 26585 raft_consensus.cc:374] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:42.128329 26585 raft_consensus.cc:729] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cb012602ec44f8e9967e83057382721, State: Initialized, Role: FOLLOWER
I20260812 07:09:42.128367 25524 catalog_manager.cc:4841] TS 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065): tablet 012018c97d184d758cb8239fc82027f2 (table test-table [id=90bdc54f0f264e478041c58830eae6fd]) successfully deleted
I20260812 07:09:42.128568 26585 consensus_queue.cc:260] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.129251 26589 tablet_bootstrap.cc:492] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: No bootstrap required, opened a new log
I20260812 07:09:42.129417 26589 ts_tablet_manager.cc:1397] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260812 07:09:42.129637 26585 ts_tablet_manager.cc:1428] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:09:42.129730 26586 tablet_bootstrap.cc:492] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Bootstrap starting.
I20260812 07:09:42.129846 26566 ts_tablet_manager.cc:1918] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:09:42.130013 26566 log.cc:1196] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/012018c97d184d758cb8239fc82027f2
I20260812 07:09:42.130117 26589 raft_consensus.cc:348] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.130357 26589 raft_consensus.cc:374] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:42.130431 26589 raft_consensus.cc:729] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Initialized, Role: FOLLOWER
I20260812 07:09:42.130470 26566 ts_tablet_manager.cc:1939] T 012018c97d184d758cb8239fc82027f2 P e6719daae3be4f439b73e12070c10089: Deleting consensus metadata
I20260812 07:09:42.130666 26589 consensus_queue.cc:260] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.131513 25522 catalog_manager.cc:4841] TS e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115): tablet 012018c97d184d758cb8239fc82027f2 (table test-table [id=90bdc54f0f264e478041c58830eae6fd]) successfully deleted
I20260812 07:09:42.131587 26586 tablet_bootstrap.cc:654] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:42.131920 26589 ts_tablet_manager.cc:1428] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:09:42.134222 26586 tablet_bootstrap.cc:492] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: No bootstrap required, opened a new log
I20260812 07:09:42.134498 26586 ts_tablet_manager.cc:1397] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260812 07:09:42.135284 26586 raft_consensus.cc:348] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.135494 26586 raft_consensus.cc:374] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:42.135583 26586 raft_consensus.cc:729] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6719daae3be4f439b73e12070c10089, State: Initialized, Role: FOLLOWER
I20260812 07:09:42.135804 26586 consensus_queue.cc:260] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.136313 26586 ts_tablet_manager.cc:1428] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260812 07:09:42.286228 26596 raft_consensus.cc:482] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:09:42.286448 26596 raft_consensus.cc:504] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.287226 26596 leader_election.cc:288] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:42.287748 25839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847d79be5e70489b997db63ed8121e30" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb012602ec44f8e9967e83057382721" is_pre_election: true
I20260812 07:09:42.287869 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847d79be5e70489b997db63ed8121e30" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089" is_pre_election: true
I20260812 07:09:42.288038 25839 raft_consensus.cc:2454] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 0.
I20260812 07:09:42.288190 26298 raft_consensus.cc:2454] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 0.
I20260812 07:09:42.288573 25954 leader_election.cc:302] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, 9cb012602ec44f8e9967e83057382721; no voters:
I20260812 07:09:42.288892 26596 raft_consensus.cc:2781] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:09:42.289021 26596 raft_consensus.cc:482] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:09:42.289119 26596 raft_consensus.cc:3037] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:42.291357 26596 raft_consensus.cc:504] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.291932 26596 leader_election.cc:288] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 election: Requested vote from peers 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065), e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115)
I20260812 07:09:42.292364 25839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847d79be5e70489b997db63ed8121e30" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9cb012602ec44f8e9967e83057382721"
I20260812 07:09:42.292569 25839 raft_consensus.cc:3037] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:42.292526 26298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "847d79be5e70489b997db63ed8121e30" candidate_uuid: "1f369ba0672b4158b4e4fc9419f35d80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6719daae3be4f439b73e12070c10089"
I20260812 07:09:42.292716 26298 raft_consensus.cc:3037] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:42.295580 25839 raft_consensus.cc:2454] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 1.
I20260812 07:09:42.295720 26298 raft_consensus.cc:2454] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f369ba0672b4158b4e4fc9419f35d80 in term 1.
I20260812 07:09:42.296118 25954 leader_election.cc:302] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f369ba0672b4158b4e4fc9419f35d80, 9cb012602ec44f8e9967e83057382721; no voters:
I20260812 07:09:42.296408 26596 raft_consensus.cc:2781] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:42.296831 26596 raft_consensus.cc:686] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 LEADER]: Becoming Leader. State: Replica: 1f369ba0672b4158b4e4fc9419f35d80, State: Running, Role: LEADER
I20260812 07:09:42.297125 26596 consensus_queue.cc:237] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } }
I20260812 07:09:42.299649 25546 catalog_manager.cc:5495] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "1f369ba0672b4158b4e4fc9419f35d80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f369ba0672b4158b4e4fc9419f35d80" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 } health_report { overall_health: UNKNOWN } } }
I20260812 07:09:42.336962 26599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:42.331720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.21.181.66:39811)
0812 07:09:42.331908 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:42.331914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:42.331962 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:09:42.332159 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:42.332164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:42.332246 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:42.332383 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:09:42.332659 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.333711 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.335030 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:42.335646 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:42.335782 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:42.336235 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:09:42.336381 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:09:42.336481 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:42.336614 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 07:09:42.336693 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:09:42.345396 26298 raft_consensus.cc:1261] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Refusing update from remote peer 1f369ba0672b4158b4e4fc9419f35d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:42.345665 25839 raft_consensus.cc:1261] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Refusing update from remote peer 1f369ba0672b4158b4e4fc9419f35d80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:09:42.346347 26625 consensus_queue.cc:1035] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9cb012602ec44f8e9967e83057382721" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:09:42.346928 26596 consensus_queue.cc:1035] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6719daae3be4f439b73e12070c10089" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:09:42.367877 26543 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260812 07:09:43.966390 25550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:43.966630 25550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:09:43.970394 25550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd4825140f544737a88cff0aad834639: Sending DeleteTablet for 3 replicas of tablet 847d79be5e70489b997db63ed8121e30
I20260812 07:09:43.971040 25803 tablet_service.cc:1514] Processing DeleteTablet for tablet 847d79be5e70489b997db63ed8121e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:43 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:60996
I20260812 07:09:43.971194 26272 tablet_service.cc:1514] Processing DeleteTablet for tablet 847d79be5e70489b997db63ed8121e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:43 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58372
I20260812 07:09:43.971736 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25680
I20260812 07:09:43.973395 26689 tablet_replica.cc:330] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: stopping tablet replica
I20260812 07:09:43.973588 26689 raft_consensus.cc:2229] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:43.973724 26688 tablet_replica.cc:330] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: stopping tablet replica
I20260812 07:09:43.973820 26689 raft_consensus.cc:2258] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:43.973942 26688 raft_consensus.cc:2229] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:09:43.974730 26689 ts_tablet_manager.cc:1905] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:43.971079 26045 tablet_service.cc:1514] Processing DeleteTablet for tablet 847d79be5e70489b997db63ed8121e30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:09:43 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:58118
I20260812 07:09:43.975082 26688 raft_consensus.cc:2258] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:43.976018 26688 ts_tablet_manager.cc:1905] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:43.981076 26689 ts_tablet_manager.cc:1918] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:09:43.981215 26689 log.cc:1196] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/847d79be5e70489b997db63ed8121e30
I20260812 07:09:43.981657 26689 ts_tablet_manager.cc:1939] T 847d79be5e70489b997db63ed8121e30 P e6719daae3be4f439b73e12070c10089: Deleting consensus metadata
I20260812 07:09:43.982709 25522 catalog_manager.cc:4841] TS e6719daae3be4f439b73e12070c10089 (127.21.181.67:45115): tablet 847d79be5e70489b997db63ed8121e30 (table test-table [id=317b74fda3214626a81589c59d54c573]) successfully deleted
I20260812 07:09:43.982895 26695 tablet_replica.cc:330] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: stopping tablet replica
I20260812 07:09:43.983100 26695 raft_consensus.cc:2229] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:09:43.983196 26688 ts_tablet_manager.cc:1918] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:09:43.983330 26688 log.cc:1196] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/847d79be5e70489b997db63ed8121e30
I20260812 07:09:43.983500 26695 raft_consensus.cc:2258] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:09:43.983809 26688 ts_tablet_manager.cc:1939] T 847d79be5e70489b997db63ed8121e30 P 9cb012602ec44f8e9967e83057382721: Deleting consensus metadata
I20260812 07:09:43.984468 26695 ts_tablet_manager.cc:1905] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:09:43.984925 25524 catalog_manager.cc:4841] TS 9cb012602ec44f8e9967e83057382721 (127.21.181.65:34065): tablet 847d79be5e70489b997db63ed8121e30 (table test-table [id=317b74fda3214626a81589c59d54c573]) successfully deleted
I20260812 07:09:43.989260 26695 ts_tablet_manager.cc:1918] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260812 07:09:43.989374 26695 log.cc:1196] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/847d79be5e70489b997db63ed8121e30
I20260812 07:09:43.989677 26695 ts_tablet_manager.cc:1939] T 847d79be5e70489b997db63ed8121e30 P 1f369ba0672b4158b4e4fc9419f35d80: Deleting consensus metadata
I20260812 07:09:43.990432 25522 catalog_manager.cc:4841] TS 1f369ba0672b4158b4e4fc9419f35d80 (127.21.181.66:39811): tablet 847d79be5e70489b997db63ed8121e30 (table test-table [id=317b74fda3214626a81589c59d54c573]) successfully deleted
Aug 12 07:09:43 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:43.999782 26691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:43.974267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51480 (local address 127.21.181.65:34065)
0812 07:09:43.977125 (+ 2858us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:43.977131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:43.981982 (+ 4851us) server_negotiation.cc:407] Connection header received
0812 07:09:43.982048 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:43.982054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:43.982134 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:43.982238 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:43.984014 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:43.985031 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:43.986331 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:43.986798 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:43.986905 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:43.994919 (+ 8014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:43.994934 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:43.994952 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:43.994982 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:43.997546 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:43.998159 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:43.998166 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:43.998168 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:43.998293 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:43.998572 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:43.998575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:43.998577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:43.998905 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:43.999121 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:43.999312 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 07:09:43.999467 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2698,"thread_start_us":2568,"threads_started":1}
W20260812 07:09:44.000459 25916 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51480
W20260812 07:09:44.001022 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25680: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:44.001211 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25680
I20260812 07:09:44.036967 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25928
Aug 12 07:09:44 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:09:44.050984 26697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:44.037674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37626 (local address 127.21.181.66:39811)
0812 07:09:44.038166 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:44.038170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:44.038186 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:09:44.038239 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:44.038243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:44.038302 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:44.038388 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:44.039590 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:44.040473 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:44.041635 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:44.042065 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:44.042157 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:44.045932 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:44.045946 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:44.045962 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:44.046005 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:44.048832 (+ 2827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:44.049375 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:44.049381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:44.049383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:44.049508 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:44.049752 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:44.049755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:44.049757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:44.050081 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:44.050298 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:44.050507 (+ 209us) server_negotiation.cc:299] Negotiation successful
0812 07:09:44.050665 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":228,"threads_started":1}
W20260812 07:09:44.051606 26152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37626
W20260812 07:09:44.052201 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25928: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:44.052342 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25928
I20260812 07:09:44.083606 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26163
Aug 12 07:09:44 dist-test-slave-lgr5 krb5kdc[25456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518581, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:09:44.098433 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:44.084341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36970 (local address 127.21.181.67:45115)
0812 07:09:44.084966 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:44.084970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:44.085004 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:09:44.085059 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:44.085063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:44.085127 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:44.085227 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:44.086541 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:44.087293 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:44.088560 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:44.089007 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:44.089117 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:44.093225 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:44.093238 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:44.093254 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:44.093287 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:44.095780 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:44.096482 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:44.096490 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:44.096493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:44.096618 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:44.096968 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:44.096971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:44.096981 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:44.097327 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:44.097624 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:44.097915 (+ 291us) server_negotiation.cc:299] Negotiation successful
0812 07:09:44.098150 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":379,"threads_started":1}
W20260812 07:09:44.099126 26389 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36970
W20260812 07:09:44.099663 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26163: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:44.099800 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26163
I20260812 07:09:44.129226 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25491
I20260812 07:09:44.130275 25623 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:44.252049 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25491
2026-08-12T07:09:44Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8512 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:44 dist-test-slave-lgr5 krb5kdc[26708](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:44 dist-test-slave-lgr5 krb5kdc[26708](info): set up 2 sockets
Aug 12 07:09:44 dist-test-slave-lgr5 krb5kdc[26708](info): commencing operation
krb5kdc: starting...
W20260812 07:09:45.482226 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:45 dist-test-slave-lgr5 krb5kdc[26708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:45.655023 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44617
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:44617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:45.965343 26731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:45.965641 26731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:45.965780 26731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:45.973549 26731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:45.973670 26731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:45.973778 26731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:45.973865 26731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:45.973956 26731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:45.984457 26731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:44617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:45.985754 26731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:45.986986 26731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:45.998270 26738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:45.998342 26743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:45.998347 26741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:45.998803 26731 server_base.cc:1034] running on GCE node
I20260812 07:09:45.999570 26731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:46.000926 26731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:46.002130 26731 hybrid_clock.cc:648] HybridClock initialized: now 1786518586002087 us; error 35 us; skew 500 ppm
I20260812 07:09:46.002460 26731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:46 dist-test-slave-lgr5 krb5kdc[26708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:46.005502 26731 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:46.006865 26731 webserver.cc:460] Webserver started at http://127.21.181.126:45537/ using document root <none> and password file <none>
I20260812 07:09:46.007521 26731 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:46.007637 26731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:46.007925 26731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:46.010807 26731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "81580f11dbfc40718c6c483905eca31a"
format_stamp: "Formatted at 2026-08-12 07:09:46 on dist-test-slave-lgr5"
server_key: "a7e733366a248c36bc42d149e87a37f8"
server_key_iv: "7a2f7752390bd975cc1abd3a09a3f2fe"
server_key_version: "encryptionkey@0"
I20260812 07:09:46.011535 26731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "81580f11dbfc40718c6c483905eca31a"
format_stamp: "Formatted at 2026-08-12 07:09:46 on dist-test-slave-lgr5"
server_key: "a7e733366a248c36bc42d149e87a37f8"
server_key_iv: "7a2f7752390bd975cc1abd3a09a3f2fe"
server_key_version: "encryptionkey@0"
I20260812 07:09:46.016490 26731 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260812 07:09:46.019662 26751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:46.020872 26731 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:46.021041 26731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "81580f11dbfc40718c6c483905eca31a"
format_stamp: "Formatted at 2026-08-12 07:09:46 on dist-test-slave-lgr5"
server_key: "a7e733366a248c36bc42d149e87a37f8"
server_key_iv: "7a2f7752390bd975cc1abd3a09a3f2fe"
server_key_version: "encryptionkey@0"
I20260812 07:09:46.021214 26731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:46.049556 26731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:46.053524 26731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:46.053774 26731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:46.070484 26731 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:44617
I20260812 07:09:46.070514 26841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:44617 every 8 connection(s)
I20260812 07:09:46.071720 26731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:46.075192 26842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:46.078426 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26731
I20260812 07:09:46.078639 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:46.079267 22229 external_mini_cluster.cc:1428] Setting key 8dcd191c400ea61c9668fb63c2501dd2
I20260812 07:09:46.087306 26842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a: Bootstrap starting.
I20260812 07:09:46.090854 26842 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:46.092144 26842 log.cc:824] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:09:46 dist-test-slave-lgr5 krb5kdc[26708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:46.094883 26842 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a: No bootstrap required, opened a new log
I20260812 07:09:46.100996 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:46.081061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51490 (local address 127.21.181.126:44617)
0812 07:09:46.081772 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:46.081783 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:46.081822 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:09:46.083332 (+ 1510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:46.083353 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:46.083703 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:46.084036 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:46.085727 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:46.086966 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:46.088718 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:46.089384 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:46.089505 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:46.094113 (+ 4608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:46.094121 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:46.094134 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:46.094171 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:46.097182 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:46.097814 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:46.097820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:46.097826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:46.098009 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:46.098379 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:46.098382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:46.098384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:46.098905 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:46.099218 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:46.099641 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 07:09:46.100025 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":318,"threads_started":1}
I20260812 07:09:46.102466 26842 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81580f11dbfc40718c6c483905eca31a" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44617 } }
I20260812 07:09:46.102797 26842 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:46.102891 26842 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81580f11dbfc40718c6c483905eca31a, State: Initialized, Role: FOLLOWER
I20260812 07:09:46.103391 26842 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81580f11dbfc40718c6c483905eca31a" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44617 } }
I20260812 07:09:46.103674 26842 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:46.103801 26842 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:46.103926 26842 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:46.106022 26842 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81580f11dbfc40718c6c483905eca31a" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44617 } }
I20260812 07:09:46.106492 26842 leader_election.cc:302] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81580f11dbfc40718c6c483905eca31a; no voters:
I20260812 07:09:46.107009 26842 leader_election.cc:288] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:46.107113 26849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:46.108459 26849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [term 1 LEADER]: Becoming Leader. State: Replica: 81580f11dbfc40718c6c483905eca31a, State: Running, Role: LEADER
I20260812 07:09:46.108875 26842 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:46.109153 26849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81580f11dbfc40718c6c483905eca31a" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44617 } }
I20260812 07:09:46.111985 26853 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81580f11dbfc40718c6c483905eca31a. Latest consensus state: current_term: 1 leader_uuid: "81580f11dbfc40718c6c483905eca31a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81580f11dbfc40718c6c483905eca31a" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44617 } } }
I20260812 07:09:46.111963 26850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81580f11dbfc40718c6c483905eca31a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81580f11dbfc40718c6c483905eca31a" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44617 } } }
I20260812 07:09:46.112419 26853 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:46.112443 26850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:46.114068 26864 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:46.119264 26864 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:46.130259 26864 catalog_manager.cc:1347] Generated new cluster ID: 43ba32d48e2d47dc843f2ddcc2a94406
I20260812 07:09:46.135751 26864 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:46.151656 26864 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:46.153443 26864 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:46.178220 26864 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81580f11dbfc40718c6c483905eca31a: Generated new TSK 0
I20260812 07:09:46.179224 26864 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:46.224865 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44617
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:46.538882 26891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:46.539117 26891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:46.539248 26891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:46.546145 26891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:46.546252 26891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:46.546412 26891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:46.555503 26891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:46.556622 26891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:46.557804 26891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:46.569837 26898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:46.569860 26900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:46.570240 26897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:46.570546 26891 server_base.cc:1034] running on GCE node
I20260812 07:09:46.571092 26891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:46.571900 26891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:46.573077 26891 hybrid_clock.cc:648] HybridClock initialized: now 1786518586573009 us; error 57 us; skew 500 ppm
I20260812 07:09:46.573359 26891 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:46 dist-test-slave-lgr5 krb5kdc[26708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:46.576313 26891 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:46.577564 26891 webserver.cc:460] Webserver started at http://127.21.181.65:42379/ using document root <none> and password file <none>
I20260812 07:09:46.578190 26891 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:46.578295 26891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:46.578588 26891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:46.581322 26891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "ca30a4c27f404c9f8d2c6f3754a1475f"
format_stamp: "Formatted at 2026-08-12 07:09:46 on dist-test-slave-lgr5"
server_key: "cd122cfae79c1a0dc4bdcfba255bffad"
server_key_iv: "ac9b8b04275bf7d5801cd5c0658c9bb0"
server_key_version: "encryptionkey@0"
I20260812 07:09:46.582075 26891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "ca30a4c27f404c9f8d2c6f3754a1475f"
format_stamp: "Formatted at 2026-08-12 07:09:46 on dist-test-slave-lgr5"
server_key: "cd122cfae79c1a0dc4bdcfba255bffad"
server_key_iv: "ac9b8b04275bf7d5801cd5c0658c9bb0"
server_key_version: "encryptionkey@0"
I20260812 07:09:46.587083 26891 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260812 07:09:46.590271 26912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:46.591475 26891 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:09:46.591629 26891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "ca30a4c27f404c9f8d2c6f3754a1475f"
format_stamp: "Formatted at 2026-08-12 07:09:46 on dist-test-slave-lgr5"
server_key: "cd122cfae79c1a0dc4bdcfba255bffad"
server_key_iv: "ac9b8b04275bf7d5801cd5c0658c9bb0"
server_key_version: "encryptionkey@0"
I20260812 07:09:46.591804 26891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:46.615968 26891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:46.619458 26891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:46.619695 26891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:46.620582 26891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:46.621788 26891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:46.621860 26891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:46.621973 26891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:46.622031 26891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:46.651051 26891 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:35495
I20260812 07:09:46.651108 27120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:35495 every 8 connection(s)
I20260812 07:09:46.652220 26891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:46.660964 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26891
I20260812 07:09:46.661166 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:46.661758 22229 external_mini_cluster.cc:1428] Setting key e73806d0cdb63027ee97e5900f71d587
Aug 12 07:09:46 dist-test-slave-lgr5 krb5kdc[26708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518586, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:46.670270 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:46.654438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:41679 (local address 127.21.181.126:44617)
0812 07:09:46.654861 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:46.654865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:46.655621 (+ 756us) server_negotiation.cc:407] Connection header received
0812 07:09:46.657433 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:46.657439 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:46.657519 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:46.657613 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:46.659015 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:46.659735 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:46.661289 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:46.661693 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:46.661804 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:46.665075 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:46.665080 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:46.665083 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:46.665116 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:46.667438 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:46.668128 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:46.668132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:46.668134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:46.668254 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:46.668662 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:46.668666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:46.668668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:46.668931 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:46.669070 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:46.669698 (+ 628us) server_negotiation.cc:299] Negotiation successful
0812 07:09:46.669910 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":227,"threads_started":1}
I20260812 07:09:46.671060 27129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:46.654794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44617 (local address 127.21.181.65:41679)
0812 07:09:46.655451 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:46.655506 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:46.657174 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:46.657842 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:46.657848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:46.658309 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:46.658841 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:46.658858 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:46.659878 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:46.659882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:46.661140 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:46.661154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:46.661902 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:46.661910 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:46.662134 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:46.662743 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:46.662766 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:46.664865 (+ 2099us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:46.667571 (+ 2706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:46.667579 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:46.667593 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:46.667975 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:46.668357 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:46.668361 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:46.668365 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:46.668551 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:46.669065 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:46.669072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:46.669351 (+ 279us) client_negotiation.cc:769] Sending connection context
0812 07:09:46.669579 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 07:09:46.670037 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":262,"threads_started":1}
I20260812 07:09:46.672451 27124 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44617
I20260812 07:09:46.672740 27124 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:46.673484 27124 heartbeater.cc:511] Master 127.21.181.126:44617 requested a full tablet report, sending...
I20260812 07:09:46.675786 26786 ts_manager.cc:194] Registered new tserver with Master: ca30a4c27f404c9f8d2c6f3754a1475f (127.21.181.65:35495)
I20260812 07:09:46.677675 26786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:41679
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:46.706768 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44617
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:47.017668 27135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:47.017925 27135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:47.018029 27135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:47.025149 27135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:47.025246 27135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:47.025362 27135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:47.034601 27135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:47.035764 27135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:47.036993 27135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:47.049119 27144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:47.049187 27145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:47.049220 27150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:47.049958 27135 server_base.cc:1034] running on GCE node
I20260812 07:09:47.050501 27135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:47.051342 27135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:47.052536 27135 hybrid_clock.cc:648] HybridClock initialized: now 1786518587052480 us; error 49 us; skew 500 ppm
I20260812 07:09:47.052819 27135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[26708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518587, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:47.055670 27135 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:47.057003 27135 webserver.cc:460] Webserver started at http://127.21.181.66:45477/ using document root <none> and password file <none>
I20260812 07:09:47.057631 27135 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:47.057745 27135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:47.058032 27135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:47.060766 27135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "f59916982a25400fbb1c12a620ea9a94"
format_stamp: "Formatted at 2026-08-12 07:09:47 on dist-test-slave-lgr5"
server_key: "6abaf27cbdba64cb5ce9c7dd438d2114"
server_key_iv: "9e3f41d4287a782405224c4302f651cf"
server_key_version: "encryptionkey@0"
I20260812 07:09:47.061533 27135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "f59916982a25400fbb1c12a620ea9a94"
format_stamp: "Formatted at 2026-08-12 07:09:47 on dist-test-slave-lgr5"
server_key: "6abaf27cbdba64cb5ce9c7dd438d2114"
server_key_iv: "9e3f41d4287a782405224c4302f651cf"
server_key_version: "encryptionkey@0"
I20260812 07:09:47.066236 27135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:47.069361 27159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:47.070614 27135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:09:47.070768 27135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "f59916982a25400fbb1c12a620ea9a94"
format_stamp: "Formatted at 2026-08-12 07:09:47 on dist-test-slave-lgr5"
server_key: "6abaf27cbdba64cb5ce9c7dd438d2114"
server_key_iv: "9e3f41d4287a782405224c4302f651cf"
server_key_version: "encryptionkey@0"
I20260812 07:09:47.070936 27135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:47.097241 27135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:47.100843 27135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:47.101075 27135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:47.101902 27135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:47.103147 27135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:47.103219 27135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:47.103308 27135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:47.103350 27135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:47.130975 27135 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:43847
I20260812 07:09:47.131034 27360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:43847 every 8 connection(s)
I20260812 07:09:47.132160 27135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:47.132683 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27135
I20260812 07:09:47.132887 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:47.133425 22229 external_mini_cluster.cc:1428] Setting key 4090d85697904ee176c3edf769a70b3e
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[26708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518587, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:47.152624 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:47.134941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:57731 (local address 127.21.181.126:44617)
0812 07:09:47.135149 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:47.135155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:47.136330 (+ 1175us) server_negotiation.cc:407] Connection header received
0812 07:09:47.138116 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:47.138121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:47.138198 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:47.138284 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:47.139855 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.140948 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.143028 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.143444 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.143548 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:47.147120 (+ 3572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:47.147125 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:47.147129 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:47.147160 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:47.149656 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:47.150434 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:47.150437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:47.150440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:47.150559 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:47.151005 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:47.151008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:47.151010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:47.151240 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:47.151381 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:47.152057 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 07:09:47.152306 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:09:47.153522 27366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:47.135397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44617 (local address 127.21.181.66:57731)
0812 07:09:47.136185 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:47.136231 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:47.137870 (+ 1639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:47.138529 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:47.138536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:47.139003 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:47.139673 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:47.139701 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.141156 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.141162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:47.142884 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:47.142900 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.143659 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.143665 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:47.143857 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:47.144480 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:47.144503 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:47.146901 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:47.149788 (+ 2887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:47.149796 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:47.149809 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:47.150341 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:47.150651 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:47.150655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:47.150658 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:47.150901 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:47.151371 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:47.151379 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:47.151691 (+ 312us) client_negotiation.cc:769] Sending connection context
0812 07:09:47.151925 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 07:09:47.152429 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":306,"threads_started":1}
I20260812 07:09:47.154913 27362 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44617
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
I20260812 07:09:47.155259 27362 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:47.155988 27362 heartbeater.cc:511] Master 127.21.181.126:44617 requested a full tablet report, sending...
I20260812 07:09:47.157430 26786 ts_manager.cc:194] Registered new tserver with Master: f59916982a25400fbb1c12a620ea9a94 (127.21.181.66:43847)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
I20260812 07:09:47.158221 26786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:57731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:47.178377 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44617
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:38349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:47.487543 27371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:47.487792 27371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:47.487977 27371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:47.494984 27371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:47.495088 27371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:47.495205 27371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:47.504335 27371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:47.505415 27371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:47.506572 27371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:47.518802 27387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:47.518848 27384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:47.518997 27382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:47.519624 27371 server_base.cc:1034] running on GCE node
I20260812 07:09:47.520185 27371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:47.520982 27371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:47.522178 27371 hybrid_clock.cc:648] HybridClock initialized: now 1786518587522132 us; error 40 us; skew 500 ppm
I20260812 07:09:47.522464 27371 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[26708](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518587, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:47.525344 27371 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:47.526705 27371 webserver.cc:460] Webserver started at http://127.21.181.67:35371/ using document root <none> and password file <none>
I20260812 07:09:47.527307 27371 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:47.527427 27371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:47.527717 27371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:47.530419 27371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "1db43a2c39874eeba3699ca8f2cba131"
format_stamp: "Formatted at 2026-08-12 07:09:47 on dist-test-slave-lgr5"
server_key: "15a8005f734d7be8077ac20f3044ca77"
server_key_iv: "dca9fce61fd086771962f13dcc6ca88c"
server_key_version: "encryptionkey@0"
I20260812 07:09:47.531153 27371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "1db43a2c39874eeba3699ca8f2cba131"
format_stamp: "Formatted at 2026-08-12 07:09:47 on dist-test-slave-lgr5"
server_key: "15a8005f734d7be8077ac20f3044ca77"
server_key_iv: "dca9fce61fd086771962f13dcc6ca88c"
server_key_version: "encryptionkey@0"
I20260812 07:09:47.535810 27371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260812 07:09:47.538978 27399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:47.540213 27371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:47.540361 27371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "1db43a2c39874eeba3699ca8f2cba131"
format_stamp: "Formatted at 2026-08-12 07:09:47 on dist-test-slave-lgr5"
server_key: "15a8005f734d7be8077ac20f3044ca77"
server_key_iv: "dca9fce61fd086771962f13dcc6ca88c"
server_key_version: "encryptionkey@0"
I20260812 07:09:47.540555 27371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:47.558980 27371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:47.562423 27371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:47.562680 27371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:47.563503 27371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:47.564817 27371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:47.564894 27371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:47.565021 27371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:47.565073 27371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:47.595142 27371 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:43985
I20260812 07:09:47.595187 27605 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:43985 every 8 connection(s)
I20260812 07:09:47.596499 27371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:47.604421 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27371
I20260812 07:09:47.604594 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:47.604995 22229 external_mini_cluster.cc:1428] Setting key 3f822a75596751c22d50e8251a6ee05d
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[26708](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518587, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:47.614506 27127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:47.598841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:46543 (local address 127.21.181.126:44617)
0812 07:09:47.599062 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:47.599069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:47.600162 (+ 1093us) server_negotiation.cc:407] Connection header received
0812 07:09:47.601856 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:47.601860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:47.601925 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:47.602015 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:47.603308 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.604031 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.605612 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.606136 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.606259 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:47.609738 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:47.609742 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:47.609746 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:47.609776 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:47.612015 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:47.612664 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:47.612667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:47.612669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:47.612783 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:47.613141 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:47.613144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:47.613146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:47.613420 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:47.613528 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:47.614101 (+ 573us) server_negotiation.cc:299] Negotiation successful
0812 07:09:47.614273 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:09:47.615350 27608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:47.599290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44617 (local address 127.21.181.67:46543)
0812 07:09:47.599989 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:47.600028 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:47.601625 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:47.602189 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:47.602196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:47.602660 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:47.603186 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:47.603204 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.604195 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.604199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:47.605471 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:47.605483 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:47.606362 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:47.606370 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:47.606650 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:47.607300 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:47.607328 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:47.609511 (+ 2183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:47.612156 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:47.612163 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:47.612173 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:47.612574 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:47.612856 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:47.612859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:47.612861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:47.613045 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:47.613516 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:47.613521 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:47.613783 (+ 262us) client_negotiation.cc:769] Sending connection context
0812 07:09:47.613998 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 07:09:47.614348 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":263,"threads_started":1}
I20260812 07:09:47.616591 27606 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44617
I20260812 07:09:47.616858 27606 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:47.617519 27606 heartbeater.cc:511] Master 127.21.181.126:44617 requested a full tablet report, sending...
I20260812 07:09:47.618898 26786 ts_manager.cc:194] Registered new tserver with Master: 1db43a2c39874eeba3699ca8f2cba131 (127.21.181.67:43985)
I20260812 07:09:47.619676 26786 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:46543
I20260812 07:09:47.620069 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:09:47.652915 27127 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:09:47.650508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.21.181.126:44617)
0812 07:09:47.650695 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:47.650701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:47.651179 (+ 478us) server_negotiation.cc:407] Connection header received
0812 07:09:47.651317 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:47.651322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:47.651431 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:47.651552 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:47.652681 (+ 1129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51504: BlockingRecv error: recv got EOF from 127.0.0.1:51504 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:09:47.656222 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26891
W20260812 07:09:47.658694 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26891: Not authorized: Client connection negotiation failed: client connection to 127.21.181.65:35495: server requires authentication, but client does not have Kerberos credentials available
I20260812 07:09:47.658850 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26891
I20260812 07:09:47.686201 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27135
W20260812 07:09:47.688472 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27135: Not authorized: Client connection negotiation failed: client connection to 127.21.181.66:43847: server requires authentication, but client does not have Kerberos credentials available
I20260812 07:09:47.688647 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27135
I20260812 07:09:47.711015 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27371
W20260812 07:09:47.713795 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27371: Not authorized: Client connection negotiation failed: client connection to 127.21.181.67:43985: server requires authentication, but client does not have Kerberos credentials available
I20260812 07:09:47.713912 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27371
I20260812 07:09:47.729892 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26731
I20260812 07:09:47.730902 26834 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:47.832147 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26731
2026-08-12T07:09:47Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3581 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[27635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[27635](info): set up 2 sockets
Aug 12 07:09:47 dist-test-slave-lgr5 krb5kdc[27635](info): commencing operation
krb5kdc: starting...
W20260812 07:09:49.053714 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:49 dist-test-slave-lgr5 krb5kdc[27635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:49.247008 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:36099
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:39459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:36099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:49.556866 27662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:49.557117 27662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:49.557210 27662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:49.564116 27662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:49.564206 27662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:49.564257 27662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:49.564311 27662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:49.564355 27662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:49.573563 27662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:39459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:36099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:36099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:49.574580 27662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:49.575732 27662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:49.586565 27675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:49.586534 27676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:49.586481 27678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:49.587700 27662 server_base.cc:1034] running on GCE node
I20260812 07:09:49.588438 27662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:49.589632 27662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:49.590829 27662 hybrid_clock.cc:648] HybridClock initialized: now 1786518589590785 us; error 38 us; skew 500 ppm
I20260812 07:09:49.591169 27662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:49 dist-test-slave-lgr5 krb5kdc[27635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518589, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:49.598382 27662 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:49.599743 27662 webserver.cc:460] Webserver started at http://127.21.181.126:43875/ using document root <none> and password file <none>
I20260812 07:09:49.600426 27662 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:49.600551 27662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:49.600840 27662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:49.603744 27662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "9ffb18c52ae94e5d8ef47713182d8afa"
format_stamp: "Formatted at 2026-08-12 07:09:49 on dist-test-slave-lgr5"
server_key: "e57705dc85ccbf91b0e79373f647542d"
server_key_iv: "304f428acb9152b530a55fd39a04c20f"
server_key_version: "encryptionkey@0"
I20260812 07:09:49.604521 27662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "9ffb18c52ae94e5d8ef47713182d8afa"
format_stamp: "Formatted at 2026-08-12 07:09:49 on dist-test-slave-lgr5"
server_key: "e57705dc85ccbf91b0e79373f647542d"
server_key_iv: "304f428acb9152b530a55fd39a04c20f"
server_key_version: "encryptionkey@0"
I20260812 07:09:49.609259 27662 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:09:49.612470 27689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:49.613682 27662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:09:49.613831 27662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "9ffb18c52ae94e5d8ef47713182d8afa"
format_stamp: "Formatted at 2026-08-12 07:09:49 on dist-test-slave-lgr5"
server_key: "e57705dc85ccbf91b0e79373f647542d"
server_key_iv: "304f428acb9152b530a55fd39a04c20f"
server_key_version: "encryptionkey@0"
I20260812 07:09:49.613996 27662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:49.654163 27662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:49.657702 27662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:49.657945 27662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:49.674556 27662 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:36099
I20260812 07:09:49.674597 27775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:36099 every 8 connection(s)
I20260812 07:09:49.675781 27662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:49.679299 27776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:49.682606 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27662
I20260812 07:09:49.682842 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:49.683669 22229 external_mini_cluster.cc:1428] Setting key cf5d2ff6afe695bb9acdb959dc6d7e07
I20260812 07:09:49.691672 27776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa: Bootstrap starting.
I20260812 07:09:49.695245 27776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:49.696496 27776 log.cc:824] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:09:49 dist-test-slave-lgr5 krb5kdc[27635](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:49.699123 27776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa: No bootstrap required, opened a new log
I20260812 07:09:49.705431 27785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:49.685434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39566 (local address 127.21.181.126:36099)
0812 07:09:49.686188 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:49.686198 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:49.686236 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:09:49.687834 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:49.687856 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:49.688261 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:49.688570 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:49.690338 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:49.691639 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:49.693329 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:49.693961 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:49.694100 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:49.698639 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:49.698647 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:49.698659 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:49.698696 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:49.701589 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:49.702233 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:49.702239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:49.702245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:49.702429 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:49.702816 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:49.702820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:49.702822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:49.703364 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:49.703658 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:49.704120 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 07:09:49.704487 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":345,"threads_started":1}
I20260812 07:09:49.706559 27776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36099 } }
I20260812 07:09:49.706892 27776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:49.706996 27776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffb18c52ae94e5d8ef47713182d8afa, State: Initialized, Role: FOLLOWER
I20260812 07:09:49.707505 27776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36099 } }
I20260812 07:09:49.707749 27776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:49.707870 27776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:49.708038 27776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:49.708951 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:09:49.710233 27776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36099 } }
I20260812 07:09:49.710664 27776 leader_election.cc:302] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ffb18c52ae94e5d8ef47713182d8afa; no voters:
I20260812 07:09:49.711213 27776 leader_election.cc:288] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:49.711309 27791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:49.713366 27776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:49.717916 27791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [term 1 LEADER]: Becoming Leader. State: Replica: 9ffb18c52ae94e5d8ef47713182d8afa, State: Running, Role: LEADER
I20260812 07:09:49.718468 27791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36099 } }
I20260812 07:09:49.729245 27795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ffb18c52ae94e5d8ef47713182d8afa. Latest consensus state: current_term: 1 leader_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36099 } } }
I20260812 07:09:49.729622 27795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:49.738662 27824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:49.738579 27792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ffb18c52ae94e5d8ef47713182d8afa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36099 } } }
I20260812 07:09:49.738847 27792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:49.743675 27824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:49.753331 27824 catalog_manager.cc:1347] Generated new cluster ID: 8a344f2b03e54f7c8af6fabfdb3abab2
I20260812 07:09:49.753436 27824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:49.761346 27785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:49.754200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39572 (local address 127.21.181.126:36099)
0812 07:09:49.754405 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:49.754412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:49.754969 (+ 557us) server_negotiation.cc:407] Connection header received
0812 07:09:49.755175 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:49.755179 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:49.755245 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:49.755342 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:49.756385 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:49.757081 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:49.758325 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:49.758749 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:49.758857 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:49.759871 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:49.759876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:49.759880 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:09:49.760586 (+ 706us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:49.760648 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:09:49.760676 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:49.760768 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:49.760923 (+ 155us) server_negotiation.cc:299] Negotiation successful
0812 07:09:49.761014 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:09:49.770864 27824 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:49.772256 27824 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:49.778697 27824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ffb18c52ae94e5d8ef47713182d8afa: Generated new TSK 0
I20260812 07:09:49.779398 27824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:09:49.788162 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27662
I20260812 07:09:49.789450 27768 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:49.875830 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27662
2026-08-12T07:09:49Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2040 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:49 dist-test-slave-lgr5 krb5kdc[27848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:49 dist-test-slave-lgr5 krb5kdc[27848](info): set up 2 sockets
Aug 12 07:09:49 dist-test-slave-lgr5 krb5kdc[27848](info): commencing operation
krb5kdc: starting...
W20260812 07:09:51.095638 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:51 dist-test-slave-lgr5 krb5kdc[27848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:51.282858 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34681
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:34681
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:51.601011 27875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:51.601306 27875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:51.601452 27875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:51.608480 27875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:51.608594 27875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:51.608682 27875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:51.608769 27875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:51.608839 27875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:51.618330 27875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:34681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:51.619540 27875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:51.620822 27875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:51.632218 27883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:51.632304 27882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:51.632184 27886 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:51.633378 27875 server_base.cc:1034] running on GCE node
I20260812 07:09:51.634068 27875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:51.635349 27875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:51.636555 27875 hybrid_clock.cc:648] HybridClock initialized: now 1786518591636505 us; error 33 us; skew 500 ppm
I20260812 07:09:51.636868 27875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:51 dist-test-slave-lgr5 krb5kdc[27848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518591, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:51.639761 27875 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:51.641105 27875 webserver.cc:460] Webserver started at http://127.21.181.126:38179/ using document root <none> and password file <none>
I20260812 07:09:51.641726 27875 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:51.641844 27875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:51.642125 27875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:51.644937 27875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "d2fb11e11e58431db8a2bd4a65d86777"
format_stamp: "Formatted at 2026-08-12 07:09:51 on dist-test-slave-lgr5"
server_key: "54c295b754d21a17b27a5130ba2a5e13"
server_key_iv: "c5afd1f99e759747b23e31f58f3668d7"
server_key_version: "encryptionkey@0"
I20260812 07:09:51.645650 27875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "d2fb11e11e58431db8a2bd4a65d86777"
format_stamp: "Formatted at 2026-08-12 07:09:51 on dist-test-slave-lgr5"
server_key: "54c295b754d21a17b27a5130ba2a5e13"
server_key_iv: "c5afd1f99e759747b23e31f58f3668d7"
server_key_version: "encryptionkey@0"
I20260812 07:09:51.650405 27875 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:09:51.653551 27899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:51.654716 27875 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:51.654870 27875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "d2fb11e11e58431db8a2bd4a65d86777"
format_stamp: "Formatted at 2026-08-12 07:09:51 on dist-test-slave-lgr5"
server_key: "54c295b754d21a17b27a5130ba2a5e13"
server_key_iv: "c5afd1f99e759747b23e31f58f3668d7"
server_key_version: "encryptionkey@0"
I20260812 07:09:51.655035 27875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:51.675889 27875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:51.679370 27875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:51.679623 27875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:51.696198 27875 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:34681
I20260812 07:09:51.696223 27993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:34681 every 8 connection(s)
I20260812 07:09:51.697372 27875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:51.700709 27994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:51.707525 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27875
I20260812 07:09:51.707702 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:51.708351 22229 external_mini_cluster.cc:1428] Setting key 7ee8bf9d7ef8303d98507b1a90007439
I20260812 07:09:51.710333 27994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777: Bootstrap starting.
I20260812 07:09:51.713636 27994 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:51.714972 27994 log.cc:824] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:51.718190 27994 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777: No bootstrap required, opened a new log
Aug 12 07:09:51 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:51.728434 27994 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fb11e11e58431db8a2bd4a65d86777" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34681 } }
I20260812 07:09:51.728896 27994 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:51.729045 27994 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2fb11e11e58431db8a2bd4a65d86777, State: Initialized, Role: FOLLOWER
I20260812 07:09:51.729705 27994 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fb11e11e58431db8a2bd4a65d86777" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34681 } }
I20260812 07:09:51.730065 27994 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:51.730265 27994 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:51.730518 27994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:51.732280 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:51.710084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58068 (local address 127.21.181.126:34681)
0812 07:09:51.710780 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:51.710791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:51.710830 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:09:51.712531 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:51.712554 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:51.712890 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:51.713190 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:51.715144 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:51.716506 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:51.718294 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:51.718969 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:51.719090 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:51.723659 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:51.723670 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:51.723688 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:51.723740 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:51.727721 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:51.728378 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:51.728387 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:51.728395 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:51.728642 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:51.729020 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:51.729027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:51.729035 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:51.729749 (+ 714us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:51.730073 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:51.730423 (+ 350us) server_negotiation.cc:299] Negotiation successful
0812 07:09:51.730852 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":322,"threads_started":1}
I20260812 07:09:51.733520 27994 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fb11e11e58431db8a2bd4a65d86777" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34681 } }
I20260812 07:09:51.734048 27994 leader_election.cc:302] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2fb11e11e58431db8a2bd4a65d86777; no voters:
I20260812 07:09:51.734676 27994 leader_election.cc:288] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:51.734746 28001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:51.735505 28001 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [term 1 LEADER]: Becoming Leader. State: Replica: d2fb11e11e58431db8a2bd4a65d86777, State: Running, Role: LEADER
I20260812 07:09:51.736009 28001 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fb11e11e58431db8a2bd4a65d86777" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34681 } }
I20260812 07:09:51.736761 27994 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:51.738672 28002 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2fb11e11e58431db8a2bd4a65d86777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fb11e11e58431db8a2bd4a65d86777" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34681 } } }
I20260812 07:09:51.739037 28002 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:51.738778 28005 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2fb11e11e58431db8a2bd4a65d86777. Latest consensus state: current_term: 1 leader_uuid: "d2fb11e11e58431db8a2bd4a65d86777" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2fb11e11e58431db8a2bd4a65d86777" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34681 } } }
I20260812 07:09:51.739285 28005 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:51.741379 28014 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:51.746251 28014 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:51.755606 28014 catalog_manager.cc:1347] Generated new cluster ID: 228df389ab8b47f0b4d10c76250b962d
I20260812 07:09:51.755721 28014 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:51.789696 28014 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:51.791560 28014 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:51.800849 28014 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2fb11e11e58431db8a2bd4a65d86777: Generated new TSK 0
I20260812 07:09:51.801788 28014 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:51.860898 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34681
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:52.181845 28040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:52.182096 28040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:52.182194 28040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:52.189250 28040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:52.189345 28040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:52.189459 28040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:52.199019 28040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:52.200155 28040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:52.201383 28040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:52.213609 28047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:52.213641 28046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:52.213676 28050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:52.214275 28040 server_base.cc:1034] running on GCE node
I20260812 07:09:52.214812 28040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:52.215673 28040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:52.216861 28040 hybrid_clock.cc:648] HybridClock initialized: now 1786518592216822 us; error 32 us; skew 500 ppm
I20260812 07:09:52.217156 28040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:52 dist-test-slave-lgr5 krb5kdc[27848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:52.220062 28040 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:52.221370 28040 webserver.cc:460] Webserver started at http://127.21.181.65:38245/ using document root <none> and password file <none>
I20260812 07:09:52.221987 28040 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:52.222106 28040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:52.222388 28040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:52.225165 28040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "6510e6a33dcb4c3190432ba489ce8a43"
format_stamp: "Formatted at 2026-08-12 07:09:52 on dist-test-slave-lgr5"
server_key: "c688aa8f584cd52dc8d7ab8d5dd28fe5"
server_key_iv: "e5adfa3d382ead2d966ba75da8cc8143"
server_key_version: "encryptionkey@0"
I20260812 07:09:52.225886 28040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "6510e6a33dcb4c3190432ba489ce8a43"
format_stamp: "Formatted at 2026-08-12 07:09:52 on dist-test-slave-lgr5"
server_key: "c688aa8f584cd52dc8d7ab8d5dd28fe5"
server_key_iv: "e5adfa3d382ead2d966ba75da8cc8143"
server_key_version: "encryptionkey@0"
I20260812 07:09:52.230621 28040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:09:52.233855 28064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:52.235031 28040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:52.235184 28040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "6510e6a33dcb4c3190432ba489ce8a43"
format_stamp: "Formatted at 2026-08-12 07:09:52 on dist-test-slave-lgr5"
server_key: "c688aa8f584cd52dc8d7ab8d5dd28fe5"
server_key_iv: "e5adfa3d382ead2d966ba75da8cc8143"
server_key_version: "encryptionkey@0"
I20260812 07:09:52.235357 28040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:52.253351 28040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:52.256834 28040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:52.257102 28040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:52.257931 28040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:52.259179 28040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:52.259253 28040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:52.259368 28040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:52.259418 28040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:52.288645 28040 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:42173
I20260812 07:09:52.288690 28278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:42173 every 8 connection(s)
I20260812 07:09:52.289860 28040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:52.297456 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28040
I20260812 07:09:52.297634 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:52.298413 22229 external_mini_cluster.cc:1428] Setting key eca280a57266ff07e2fd81a777f8a5cf
Aug 12 07:09:52 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:52.309077 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:52.292271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:53427 (local address 127.21.181.126:34681)
0812 07:09:52.292714 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:52.292722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:52.293659 (+ 937us) server_negotiation.cc:407] Connection header received
0812 07:09:52.295358 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:52.295363 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:52.295437 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:52.295536 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:52.296981 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.297755 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.299778 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.300414 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.300564 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:52.303848 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:52.303853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:52.303856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:52.303887 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:52.306278 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:52.306930 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:52.306936 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:52.306939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:52.307077 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:52.307466 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:52.307470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:52.307472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:52.307744 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:52.307876 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:52.308651 (+ 775us) server_negotiation.cc:299] Negotiation successful
0812 07:09:52.308817 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":204,"threads_started":1}
I20260812 07:09:52.310092 28282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:52.292830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34681 (local address 127.21.181.65:53427)
0812 07:09:52.293513 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:52.293550 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:52.295118 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:52.295758 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:52.295766 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:52.296306 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:52.296835 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:52.296853 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.297929 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.297935 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:52.299647 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:52.299660 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.300654 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.300662 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:52.300874 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:52.301466 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:52.301516 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:52.303634 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:52.306406 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:52.306414 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:52.306428 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:52.306819 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:52.307181 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:52.307186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:52.307190 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:52.307363 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:52.307868 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:52.307875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:52.308280 (+ 405us) client_negotiation.cc:769] Sending connection context
0812 07:09:52.308573 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 07:09:52.309049 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":263,"threads_started":1}
I20260812 07:09:52.311569 28279 heartbeater.cc:347] Connected to a master server at 127.21.181.126:34681
I20260812 07:09:52.311820 28279 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:52.312542 28279 heartbeater.cc:511] Master 127.21.181.126:34681 requested a full tablet report, sending...
I20260812 07:09:52.314416 27932 ts_manager.cc:194] Registered new tserver with Master: 6510e6a33dcb4c3190432ba489ce8a43 (127.21.181.65:42173)
I20260812 07:09:52.315762 27932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:53427
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:52.342877 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34681
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:52.652397 28286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:52.652653 28286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:52.652776 28286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:52.659724 28286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:52.659818 28286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:52.659973 28286 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:52.669026 28286 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:52.670090 28286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:52.671272 28286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:52.683295 28305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:52.683332 28298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:52.683629 28296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:52.684253 28286 server_base.cc:1034] running on GCE node
I20260812 07:09:52.684787 28286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:52.685619 28286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:52.686818 28286 hybrid_clock.cc:648] HybridClock initialized: now 1786518592686766 us; error 39 us; skew 500 ppm
I20260812 07:09:52.687125 28286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:52 dist-test-slave-lgr5 krb5kdc[27848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:52.690019 28286 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:52.691291 28286 webserver.cc:460] Webserver started at http://127.21.181.66:37507/ using document root <none> and password file <none>
I20260812 07:09:52.691912 28286 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:52.692029 28286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:52.692332 28286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:52.695080 28286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "90dd08fc06b84628a867d99b08b17a3f"
format_stamp: "Formatted at 2026-08-12 07:09:52 on dist-test-slave-lgr5"
server_key: "771f5de2f338c8d8000745a452c3a50f"
server_key_iv: "580fc2a2c7c1b8449f8473f6eaeb7f86"
server_key_version: "encryptionkey@0"
I20260812 07:09:52.695883 28286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "90dd08fc06b84628a867d99b08b17a3f"
format_stamp: "Formatted at 2026-08-12 07:09:52 on dist-test-slave-lgr5"
server_key: "771f5de2f338c8d8000745a452c3a50f"
server_key_iv: "580fc2a2c7c1b8449f8473f6eaeb7f86"
server_key_version: "encryptionkey@0"
I20260812 07:09:52.700887 28286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 07:09:52.704258 28319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:52.705540 28286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:09:52.705679 28286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "90dd08fc06b84628a867d99b08b17a3f"
format_stamp: "Formatted at 2026-08-12 07:09:52 on dist-test-slave-lgr5"
server_key: "771f5de2f338c8d8000745a452c3a50f"
server_key_iv: "580fc2a2c7c1b8449f8473f6eaeb7f86"
server_key_version: "encryptionkey@0"
I20260812 07:09:52.705852 28286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:52.731856 28286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:52.735383 28286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:52.735646 28286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:52.736543 28286 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:52.737851 28286 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:52.737923 28286 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:52.738023 28286 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:52.738085 28286 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:52.767488 28286 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:42003
I20260812 07:09:52.767521 28538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:42003 every 8 connection(s)
I20260812 07:09:52.768731 28286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:52.778584 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28286
I20260812 07:09:52.778757 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:52.779302 22229 external_mini_cluster.cc:1428] Setting key 5d3577c8d912e2f22a2d6f8e78e98f25
Aug 12 07:09:52 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518592, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:52.787636 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:52.771044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:48929 (local address 127.21.181.126:34681)
0812 07:09:52.771254 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:52.771260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:52.772399 (+ 1139us) server_negotiation.cc:407] Connection header received
0812 07:09:52.774106 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:52.774110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:52.774178 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:52.774264 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:52.775606 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.776372 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.777808 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.778315 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.778414 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:52.782330 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:52.782335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:52.782338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:52.782368 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:52.784749 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:52.785533 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:52.785537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:52.785539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:52.785664 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:52.786092 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:52.786095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:52.786097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:52.786322 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:52.786429 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:52.787220 (+ 791us) server_negotiation.cc:299] Negotiation successful
0812 07:09:52.787382 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:09:52.788858 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:52.771546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34681 (local address 127.21.181.66:48929)
0812 07:09:52.772251 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:52.772298 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:52.773860 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:52.774437 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:52.774445 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:52.774919 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:52.775476 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:52.775493 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.776493 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.776497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:52.777667 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:52.777685 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:52.778557 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:52.778566 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:52.778844 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:52.779547 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:52.779575 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:52.782119 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:52.784881 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:52.784889 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:52.784903 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:52.785439 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:52.785759 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:52.785763 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:52.785766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:52.786006 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:52.786431 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:52.786438 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:52.786861 (+ 423us) client_negotiation.cc:769] Sending connection context
0812 07:09:52.787156 (+ 295us) client_negotiation.cc:240] Negotiation successful
0812 07:09:52.787623 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":262,"threads_started":1}
I20260812 07:09:52.790146 28540 heartbeater.cc:347] Connected to a master server at 127.21.181.126:34681
I20260812 07:09:52.790424 28540 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:52.791090 28540 heartbeater.cc:511] Master 127.21.181.126:34681 requested a full tablet report, sending...
I20260812 07:09:52.792527 27932 ts_manager.cc:194] Registered new tserver with Master: 90dd08fc06b84628a867d99b08b17a3f (127.21.181.66:42003)
I20260812 07:09:52.793541 27932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:48929
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:52.824587 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34681
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:53.133530 28553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:53.133775 28553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:53.133910 28553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:53.141044 28553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:53.141139 28553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:53.141297 28553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:53.150420 28553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34681
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:53.151538 28553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:53.152805 28553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:53.165400 28563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:53.165511 28566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:53.165537 28572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:53.166465 28553 server_base.cc:1034] running on GCE node
I20260812 07:09:53.167021 28553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:53.167882 28553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:53.169085 28553 hybrid_clock.cc:648] HybridClock initialized: now 1786518593169018 us; error 53 us; skew 500 ppm
I20260812 07:09:53.169374 28553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:53.172255 28553 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:53.173574 28553 webserver.cc:460] Webserver started at http://127.21.181.67:40049/ using document root <none> and password file <none>
I20260812 07:09:53.174188 28553 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:53.174302 28553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:53.174599 28553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:53.177620 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "3659502d03b7472faa3750efe6fa0c7c"
format_stamp: "Formatted at 2026-08-12 07:09:53 on dist-test-slave-lgr5"
server_key: "2dd04f9a3b0e218af9bbcbf52ee09ed7"
server_key_iv: "b9597fe07a833a2ccb634c2d3c3f5574"
server_key_version: "encryptionkey@0"
I20260812 07:09:53.178361 28553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "3659502d03b7472faa3750efe6fa0c7c"
format_stamp: "Formatted at 2026-08-12 07:09:53 on dist-test-slave-lgr5"
server_key: "2dd04f9a3b0e218af9bbcbf52ee09ed7"
server_key_iv: "b9597fe07a833a2ccb634c2d3c3f5574"
server_key_version: "encryptionkey@0"
I20260812 07:09:53.183130 28553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:09:53.186409 28583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:53.187574 28553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:09:53.187717 28553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "3659502d03b7472faa3750efe6fa0c7c"
format_stamp: "Formatted at 2026-08-12 07:09:53 on dist-test-slave-lgr5"
server_key: "2dd04f9a3b0e218af9bbcbf52ee09ed7"
server_key_iv: "b9597fe07a833a2ccb634c2d3c3f5574"
server_key_version: "encryptionkey@0"
I20260812 07:09:53.187879 28553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:53.210835 28553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:53.214334 28553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:53.214591 28553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:53.215466 28553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:53.216735 28553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:53.216813 28553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:53.216928 28553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:53.216980 28553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:53.246486 28553 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:39131
I20260812 07:09:53.246526 28799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:39131 every 8 connection(s)
I20260812 07:09:53.247660 28553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:53.250520 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28553
I20260812 07:09:53.250710 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:53.251312 22229 external_mini_cluster.cc:1428] Setting key 07fa65b011240ba0d391e1df04cab4fd
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:53.266170 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:53.249931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57385 (local address 127.21.181.126:34681)
0812 07:09:53.250097 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:53.250102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:53.251384 (+ 1282us) server_negotiation.cc:407] Connection header received
0812 07:09:53.253180 (+ 1796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:53.253185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:53.253264 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:53.253350 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:53.254774 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.255525 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.257099 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.257503 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.257611 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:53.260816 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:53.260820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:53.260823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:53.260855 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:53.263100 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.263748 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.263752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.263754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.263874 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.264362 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.264366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.264369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.264702 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:53.264842 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:53.265608 (+ 766us) server_negotiation.cc:299] Negotiation successful
0812 07:09:53.265836 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:09:53.266960 28803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:53.250445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34681 (local address 127.21.181.67:57385)
0812 07:09:53.251233 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:53.251276 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:53.252911 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:53.253590 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:53.253598 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:53.254089 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:53.254614 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:53.254632 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.255679 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.255683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:53.256952 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:53.256965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.257718 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.257725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:53.257937 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:53.258535 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:53.258557 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:53.260627 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:53.263226 (+ 2599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:53.263234 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:53.263249 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:53.263627 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:53.263984 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:53.263989 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:53.263992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:53.264222 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:53.264830 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:53.264837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:53.265190 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 07:09:53.265467 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 07:09:53.265919 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":330,"threads_started":1}
I20260812 07:09:53.268206 28801 heartbeater.cc:347] Connected to a master server at 127.21.181.126:34681
I20260812 07:09:53.268455 28801 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:53.269083 28801 heartbeater.cc:511] Master 127.21.181.126:34681 requested a full tablet report, sending...
I20260812 07:09:53.270413 27932 ts_manager.cc:194] Registered new tserver with Master: 3659502d03b7472faa3750efe6fa0c7c (127.21.181.67:39131)
I20260812 07:09:53.271181 27932 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:57385
I20260812 07:09:53.278306 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
I20260812 07:09:53.318608 28279 heartbeater.cc:503] Master 127.21.181.126:34681 was elected leader, sending a full tablet report...
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:53.332535 28281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:53.319458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58074 (local address 127.21.181.126:34681)
0812 07:09:53.319643 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:53.319650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:53.320066 (+ 416us) server_negotiation.cc:407] Connection header received
0812 07:09:53.320298 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:53.320302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:53.320360 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:53.320445 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:53.321829 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.322550 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.323794 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.324223 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.324313 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:53.328149 (+ 3836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:53.328155 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:53.328158 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:53.328187 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:53.330413 (+ 2226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.330990 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.330993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.330996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.331112 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.331421 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.331424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.331426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.331711 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:53.331894 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:53.332128 (+ 234us) server_negotiation.cc:299] Negotiation successful
0812 07:09:53.332287 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 07:09:53.333079 27932 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58074
I20260812 07:09:53.335024 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28040
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:53.350351 28814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:53.335892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36230 (local address 127.21.181.65:42173)
0812 07:09:53.336528 (+ 636us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:53.336536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:53.336561 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:09:53.336628 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:53.336638 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:53.336878 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:53.337023 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:53.338316 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.339407 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.340709 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.341295 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.341427 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:53.345279 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:53.345291 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:53.345308 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:53.345347 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:53.348036 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.348667 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.348674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.348676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.348817 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.349113 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.349116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.349118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.349451 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:53.349693 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:53.349858 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 07:09:53.350056 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":281,"threads_started":1}
W20260812 07:09:53.351516 28267 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36230
W20260812 07:09:53.352586 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28040: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:53.352775 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28040
I20260812 07:09:53.386062 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28286
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:09:53.401249 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:53.386914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.21.181.66:42003)
0812 07:09:53.387447 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:53.387454 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:53.387482 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:09:53.387547 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:53.387555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:53.387762 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:53.387903 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:53.389125 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.390175 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.391436 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.392016 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.392211 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:53.396035 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:53.396048 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:53.396061 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:53.396133 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:53.398924 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.399467 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.399473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.399476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.399612 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.399912 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.399915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.399917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.400318 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:53.400564 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:53.400741 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:09:53.400944 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":250,"threads_started":1}
W20260812 07:09:53.402485 28532 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35940
W20260812 07:09:53.403508 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28286: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:53.403658 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28286
I20260812 07:09:53.434299 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28553
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[27848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518593, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:09:53.449581 28818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:53.435243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.21.181.67:39131)
0812 07:09:53.435921 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:53.435928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:53.435956 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:09:53.436018 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:53.436024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:53.436272 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:53.436401 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:53.437701 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.438801 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.439945 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:53.440595 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:53.440685 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:53.444834 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:53.444845 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:53.444858 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:53.444893 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:53.447362 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.447929 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.447934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.447937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.448050 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:53.448348 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:53.448351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:53.448353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:53.448685 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:53.448902 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:53.449105 (+ 203us) server_negotiation.cc:299] Negotiation successful
0812 07:09:53.449296 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":362,"threads_started":1}
W20260812 07:09:53.450807 28790 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34554
W20260812 07:09:53.451781 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28553: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:09:53.451920 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28553
I20260812 07:09:53.472263 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27875
I20260812 07:09:53.473415 27982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:53.574844 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27875
2026-08-12T07:09:53Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3700 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[28827](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[28827](info): set up 2 sockets
Aug 12 07:09:53 dist-test-slave-lgr5 krb5kdc[28827](info): commencing operation
krb5kdc: starting...
W20260812 07:09:54.798187 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.179s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:54 dist-test-slave-lgr5 krb5kdc[28827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:54.981006 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43479
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:43479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:55.292405 28848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:55.292650 28848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:55.292776 28848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:55.299605 28848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:55.299696 28848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:55.299772 28848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:55.299845 28848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:55.299913 28848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:55.309101 28848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:43479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:55.310186 28848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:55.311375 28848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:55.322059 28854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:55.322014 28855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:55.322103 28857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:55.323177 28848 server_base.cc:1034] running on GCE node
I20260812 07:09:55.323827 28848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:55.325088 28848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:55.326292 28848 hybrid_clock.cc:648] HybridClock initialized: now 1786518595326239 us; error 40 us; skew 500 ppm
I20260812 07:09:55.326643 28848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:55 dist-test-slave-lgr5 krb5kdc[28827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:55.332762 28848 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:55.334095 28848 webserver.cc:460] Webserver started at http://127.21.181.126:34523/ using document root <none> and password file <none>
I20260812 07:09:55.334730 28848 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:55.334844 28848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:55.335156 28848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:55.337987 28848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "ffa4450ff1a14418933f29f72ab76cda"
format_stamp: "Formatted at 2026-08-12 07:09:55 on dist-test-slave-lgr5"
server_key: "34568bb3e997edfe6b364a7e72ecc2d0"
server_key_iv: "0bccf3412773c854c3b2c3b99aecebc3"
server_key_version: "encryptionkey@0"
I20260812 07:09:55.338733 28848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "ffa4450ff1a14418933f29f72ab76cda"
format_stamp: "Formatted at 2026-08-12 07:09:55 on dist-test-slave-lgr5"
server_key: "34568bb3e997edfe6b364a7e72ecc2d0"
server_key_iv: "0bccf3412773c854c3b2c3b99aecebc3"
server_key_version: "encryptionkey@0"
I20260812 07:09:55.343570 28848 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:09:55.346827 28867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:55.348016 28848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:09:55.348191 28848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "ffa4450ff1a14418933f29f72ab76cda"
format_stamp: "Formatted at 2026-08-12 07:09:55 on dist-test-slave-lgr5"
server_key: "34568bb3e997edfe6b364a7e72ecc2d0"
server_key_iv: "0bccf3412773c854c3b2c3b99aecebc3"
server_key_version: "encryptionkey@0"
I20260812 07:09:55.348356 28848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:55.379447 28848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:55.383471 28848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:55.383724 28848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:55.400309 28848 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:43479
I20260812 07:09:55.400331 28959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:43479 every 8 connection(s)
I20260812 07:09:55.401535 28848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:55.405011 28960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:55.406186 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28848
I20260812 07:09:55.406428 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:55.407240 22229 external_mini_cluster.cc:1428] Setting key 1e7ca199c3bdc7d4411c605458c6e8fa
I20260812 07:09:55.418923 28960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda: Bootstrap starting.
Aug 12 07:09:55 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518594, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:55.422547 28960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:55.423694 28960 log.cc:824] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:55.426755 28960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda: No bootstrap required, opened a new log
I20260812 07:09:55.429649 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:55.409236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.21.181.126:43479)
0812 07:09:55.410001 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:55.410012 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:55.410063 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:09:55.411616 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:55.411638 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:55.412000 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:55.412362 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:55.414050 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:55.415286 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:55.417069 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:55.417699 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:55.417820 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:55.422453 (+ 4633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:55.422460 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:55.422481 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:55.422516 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:55.425754 (+ 3238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:55.426512 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:55.426518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:55.426524 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:55.426681 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:55.427078 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:55.427082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:55.427084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:55.427611 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:55.427820 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:55.428279 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 07:09:55.428630 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":350,"threads_started":1}
I20260812 07:09:55.434432 28960 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4450ff1a14418933f29f72ab76cda" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43479 } }
I20260812 07:09:55.434760 28960 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:55.434861 28960 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffa4450ff1a14418933f29f72ab76cda, State: Initialized, Role: FOLLOWER
I20260812 07:09:55.435379 28960 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4450ff1a14418933f29f72ab76cda" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43479 } }
I20260812 07:09:55.435602 28960 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:55.435719 28960 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:55.435863 28960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:55.437860 28960 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4450ff1a14418933f29f72ab76cda" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43479 } }
I20260812 07:09:55.438311 28960 leader_election.cc:302] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffa4450ff1a14418933f29f72ab76cda; no voters:
I20260812 07:09:55.438832 28960 leader_election.cc:288] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:55.438935 28965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:55.439594 28965 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [term 1 LEADER]: Becoming Leader. State: Replica: ffa4450ff1a14418933f29f72ab76cda, State: Running, Role: LEADER
I20260812 07:09:55.440114 28965 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4450ff1a14418933f29f72ab76cda" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43479 } }
I20260812 07:09:55.440698 28960 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:55.442549 28966 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffa4450ff1a14418933f29f72ab76cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4450ff1a14418933f29f72ab76cda" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43479 } } }
I20260812 07:09:55.442929 28966 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:55.446578 28976 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:55.446496 28967 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffa4450ff1a14418933f29f72ab76cda. Latest consensus state: current_term: 1 leader_uuid: "ffa4450ff1a14418933f29f72ab76cda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffa4450ff1a14418933f29f72ab76cda" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43479 } } }
I20260812 07:09:55.446712 28967 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:55.451742 28976 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:55.461287 28976 catalog_manager.cc:1347] Generated new cluster ID: 158680177f9341e29b5a3060645a85c8
I20260812 07:09:55.461409 28976 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:55.499606 28976 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:55.501524 28976 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:55.514520 28976 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffa4450ff1a14418933f29f72ab76cda: Generated new TSK 0
I20260812 07:09:55.515434 28976 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:55.576668 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43479
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:55.890690 29006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:55.890935 29006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:55.891067 29006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:55.897948 29006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:55.898056 29006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:55.898205 29006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:55.907229 29006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:55.908371 29006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:55.909554 29006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:55.921710 29014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:55.921751 29015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:55.921666 29021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:55.922667 29006 server_base.cc:1034] running on GCE node
I20260812 07:09:55.923190 29006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:55.923995 29006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:55.925181 29006 hybrid_clock.cc:648] HybridClock initialized: now 1786518595925128 us; error 41 us; skew 500 ppm
I20260812 07:09:55.925469 29006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:55 dist-test-slave-lgr5 krb5kdc[28827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:55.928320 29006 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:55.929781 29006 webserver.cc:460] Webserver started at http://127.21.181.65:41299/ using document root <none> and password file <none>
I20260812 07:09:55.930402 29006 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:55.930522 29006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:55.930802 29006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:55.933501 29006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "683dde5f651b4e8dbdb7782ed4ff5696"
format_stamp: "Formatted at 2026-08-12 07:09:55 on dist-test-slave-lgr5"
server_key: "c31ab5a0ca2e40101619dc9ca59dde6a"
server_key_iv: "72c58febcc712b1331ec6d47b499c853"
server_key_version: "encryptionkey@0"
I20260812 07:09:55.934245 29006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "683dde5f651b4e8dbdb7782ed4ff5696"
format_stamp: "Formatted at 2026-08-12 07:09:55 on dist-test-slave-lgr5"
server_key: "c31ab5a0ca2e40101619dc9ca59dde6a"
server_key_iv: "72c58febcc712b1331ec6d47b499c853"
server_key_version: "encryptionkey@0"
I20260812 07:09:55.938941 29006 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:09:55.942205 29032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:55.943398 29006 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:09:55.943554 29006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "683dde5f651b4e8dbdb7782ed4ff5696"
format_stamp: "Formatted at 2026-08-12 07:09:55 on dist-test-slave-lgr5"
server_key: "c31ab5a0ca2e40101619dc9ca59dde6a"
server_key_iv: "72c58febcc712b1331ec6d47b499c853"
server_key_version: "encryptionkey@0"
I20260812 07:09:55.943727 29006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:55.962638 29006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:55.966099 29006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:55.966341 29006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:55.967195 29006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:55.968421 29006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:55.968511 29006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:55.968632 29006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:55.968685 29006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.000131 29006 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:41913
I20260812 07:09:56.000196 29277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:41913 every 8 connection(s)
I20260812 07:09:56.001313 29006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:56.003531 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29006
I20260812 07:09:56.003750 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:56.004500 22229 external_mini_cluster.cc:1428] Setting key e9309f8ae0046a3a3c33f6b68fb7f440
Aug 12 07:09:56 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518595, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:56.021236 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.003777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:47075 (local address 127.21.181.126:43479)
0812 07:09:56.004272 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:56.004278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:56.005334 (+ 1056us) server_negotiation.cc:407] Connection header received
0812 07:09:56.007445 (+ 2111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:56.007450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:56.007526 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:56.007618 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:56.009181 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.009893 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.011379 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.011785 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.011893 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.015327 (+ 3434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:56.015334 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:56.015338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:56.015377 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:56.018286 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.019013 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.019018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.019022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.019164 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.019547 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.019552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.019554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.019929 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:56.020113 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:56.020709 (+ 596us) server_negotiation.cc:299] Negotiation successful
0812 07:09:56.020927 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":223,"threads_started":1}
I20260812 07:09:56.021988 29285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.004339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43479 (local address 127.21.181.65:47075)
0812 07:09:56.005174 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:56.005224 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:56.007195 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:56.007903 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:56.007915 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:56.008467 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:56.009019 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:56.009037 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.010038 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.010042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:56.011241 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:56.011253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.012002 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.012010 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:56.012268 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.012862 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:56.012889 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:56.015120 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:56.018420 (+ 3300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:56.018426 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:56.018436 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:56.018880 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:56.019262 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:56.019265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:56.019267 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:56.019435 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:56.020046 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:56.020052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:56.020360 (+ 308us) client_negotiation.cc:769] Sending connection context
0812 07:09:56.020572 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 07:09:56.020927 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":322,"threads_started":1}
I20260812 07:09:56.023375 29281 heartbeater.cc:347] Connected to a master server at 127.21.181.126:43479
I20260812 07:09:56.023701 29281 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:56.024613 29281 heartbeater.cc:511] Master 127.21.181.126:43479 requested a full tablet report, sending...
I20260812 07:09:56.026717 28898 ts_manager.cc:194] Registered new tserver with Master: 683dde5f651b4e8dbdb7782ed4ff5696 (127.21.181.65:41913)
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
I20260812 07:09:56.028159 28898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:47075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:56.052917 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43479
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:56.369668 29290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:56.369921 29290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:56.370052 29290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:56.376969 29290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:56.377067 29290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:56.377223 29290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:09:56.386296 29290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:56.387418 29290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:56.388679 29290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:56.400504 29296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:56.401211 29301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:56.401491 29307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:56.401880 29290 server_base.cc:1034] running on GCE node
I20260812 07:09:56.402557 29290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:56.403594 29290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:56.404829 29290 hybrid_clock.cc:648] HybridClock initialized: now 1786518596404767 us; error 50 us; skew 500 ppm
I20260812 07:09:56.405182 29290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:56 dist-test-slave-lgr5 krb5kdc[28827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:56.408766 29290 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:09:56.410171 29290 webserver.cc:460] Webserver started at http://127.21.181.66:44855/ using document root <none> and password file <none>
I20260812 07:09:56.410815 29290 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:56.410930 29290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:56.411226 29290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:56.413991 29290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "d0daf03545cf4ddc91c19f83ef146d02"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-lgr5"
server_key: "2b4b61014f56b599d045b8fa2c38b231"
server_key_iv: "d6aabfc483f9276a5713fcca1e55b65c"
server_key_version: "encryptionkey@0"
I20260812 07:09:56.414741 29290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "d0daf03545cf4ddc91c19f83ef146d02"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-lgr5"
server_key: "2b4b61014f56b599d045b8fa2c38b231"
server_key_iv: "d6aabfc483f9276a5713fcca1e55b65c"
server_key_version: "encryptionkey@0"
I20260812 07:09:56.419598 29290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260812 07:09:56.422960 29320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.424329 29290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:09:56.424486 29290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "d0daf03545cf4ddc91c19f83ef146d02"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-lgr5"
server_key: "2b4b61014f56b599d045b8fa2c38b231"
server_key_iv: "d6aabfc483f9276a5713fcca1e55b65c"
server_key_version: "encryptionkey@0"
I20260812 07:09:56.424674 29290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:56.464345 29290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:56.468004 29290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:56.468297 29290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:56.469197 29290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:56.470463 29290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:56.470543 29290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.470662 29290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:56.470715 29290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.500216 29290 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:45399
I20260812 07:09:56.500257 29510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:45399 every 8 connection(s)
I20260812 07:09:56.501397 29290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:09:56.509189 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29290
I20260812 07:09:56.509379 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:09:56.509851 22229 external_mini_cluster.cc:1428] Setting key 01614b2b657c9fb3fa6f92d00612981b
Aug 12 07:09:56 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:56.521390 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.503893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:42315 (local address 127.21.181.126:43479)
0812 07:09:56.504063 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:56.504068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:56.505193 (+ 1125us) server_negotiation.cc:407] Connection header received
0812 07:09:56.506891 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:56.506896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:56.506963 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:56.507055 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:56.508499 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.509494 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.511151 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.511708 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.511842 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.515569 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:56.515574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:56.515577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:56.515608 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:56.518022 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.518840 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.518846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.518849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.519019 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.519510 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.519515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.519518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.519834 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:56.519958 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:56.520813 (+ 855us) server_negotiation.cc:299] Negotiation successful
0812 07:09:56.521055 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:09:56.522289 29514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.504371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43479 (local address 127.21.181.66:42315)
0812 07:09:56.505044 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:56.505080 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:56.506652 (+ 1572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:56.507234 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:56.507241 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:56.507747 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:56.508342 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:56.508362 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.509643 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.509646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:56.511014 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:56.511027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.511936 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.511944 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:56.512282 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.512928 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:56.512955 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:56.515363 (+ 2408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:56.518157 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:56.518165 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:56.518178 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:56.518695 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:56.519159 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:56.519163 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:56.519166 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:56.519390 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:56.519963 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:56.519970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:56.520364 (+ 394us) client_negotiation.cc:769] Sending connection context
0812 07:09:56.520697 (+ 333us) client_negotiation.cc:240] Negotiation successful
0812 07:09:56.521201 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":262,"threads_started":1}
I20260812 07:09:56.523681 29512 heartbeater.cc:347] Connected to a master server at 127.21.181.126:43479
I20260812 07:09:56.523962 29512 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:56.524688 29512 heartbeater.cc:511] Master 127.21.181.126:43479 requested a full tablet report, sending...
I20260812 07:09:56.526110 28898 ts_manager.cc:194] Registered new tserver with Master: d0daf03545cf4ddc91c19f83ef146d02 (127.21.181.66:45399)
I20260812 07:09:56.527194 28898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:42315
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:09:56.556766 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43479
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:36713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:56.865836 29517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:56.866097 29517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:56.866199 29517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:56.873028 29517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:56.873118 29517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:56.873231 29517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:09:56.882375 29517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:56.883502 29517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:56.884707 29517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:56.896919 29527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:56.896965 29530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:56.897002 29528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:56.897787 29517 server_base.cc:1034] running on GCE node
I20260812 07:09:56.898344 29517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:56.899202 29517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:56.900388 29517 hybrid_clock.cc:648] HybridClock initialized: now 1786518596900339 us; error 38 us; skew 500 ppm
I20260812 07:09:56.900660 29517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:56 dist-test-slave-lgr5 krb5kdc[28827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:56.903563 29517 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:09:56.904882 29517 webserver.cc:460] Webserver started at http://127.21.181.67:35459/ using document root <none> and password file <none>
I20260812 07:09:56.905476 29517 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:56.905578 29517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:56.905828 29517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:56.908551 29517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "3b6853df21a34832ad4a90b112f0287f"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-lgr5"
server_key: "aab459ece72d3534388accb4342d4ba3"
server_key_iv: "99b8a503884fc0e76d28fba30f38f785"
server_key_version: "encryptionkey@0"
I20260812 07:09:56.909307 29517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "3b6853df21a34832ad4a90b112f0287f"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-lgr5"
server_key: "aab459ece72d3534388accb4342d4ba3"
server_key_iv: "99b8a503884fc0e76d28fba30f38f785"
server_key_version: "encryptionkey@0"
I20260812 07:09:56.913957 29517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:56.917102 29539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.918274 29517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:09:56.918401 29517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "3b6853df21a34832ad4a90b112f0287f"
format_stamp: "Formatted at 2026-08-12 07:09:56 on dist-test-slave-lgr5"
server_key: "aab459ece72d3534388accb4342d4ba3"
server_key_iv: "99b8a503884fc0e76d28fba30f38f785"
server_key_version: "encryptionkey@0"
I20260812 07:09:56.918562 29517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:56.938735 29517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:56.942678 29517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:56.942900 29517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:56.943724 29517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:56.944928 29517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:56.945010 29517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.945101 29517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:56.945142 29517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:56.974344 29517 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:41115
I20260812 07:09:56.974404 29765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:41115 every 8 connection(s)
I20260812 07:09:56.975540 29517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:09:56.982411 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29517
I20260812 07:09:56.982584 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:09:56.983001 22229 external_mini_cluster.cc:1428] Setting key 809e73c6cd071f1e12a0e69e1e076189
Aug 12 07:09:56 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518596, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:56.993769 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.977829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:37859 (local address 127.21.181.126:43479)
0812 07:09:56.978009 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:56.978014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:56.979084 (+ 1070us) server_negotiation.cc:407] Connection header received
0812 07:09:56.980882 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:56.980886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:56.980960 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:56.981047 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:56.982451 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.983309 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.985135 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.985627 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.985736 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.988996 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:56.989002 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:56.989005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:56.989038 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:56.991279 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.991902 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.991905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.991907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.992027 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:56.992397 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:56.992400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:56.992401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:56.992687 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:56.992796 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:56.993376 (+ 580us) server_negotiation.cc:299] Negotiation successful
0812 07:09:56.993538 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:09:56.994679 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:56.978232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43479 (local address 127.21.181.67:37859)
0812 07:09:56.978908 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:56.978948 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:56.980615 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:56.981226 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:56.981233 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:56.981725 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:56.982270 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:56.982288 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.983462 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.983466 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:56.985000 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:56.985012 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:56.985845 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:56.985853 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:56.986090 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:56.986712 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:56.986735 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:56.988801 (+ 2066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:56.991404 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:56.991411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:56.991422 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:56.991814 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:56.992126 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:56.992130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:56.992134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:56.992313 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:56.992777 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:56.992782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:56.993081 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 07:09:56.993291 (+ 210us) client_negotiation.cc:240] Negotiation successful
0812 07:09:56.993634 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":260,"threads_started":1}
I20260812 07:09:56.995865 29766 heartbeater.cc:347] Connected to a master server at 127.21.181.126:43479
I20260812 07:09:56.996198 29766 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:56.997004 29766 heartbeater.cc:511] Master 127.21.181.126:43479 requested a full tablet report, sending...
I20260812 07:09:56.998386 28898 ts_manager.cc:194] Registered new tserver with Master: 3b6853df21a34832ad4a90b112f0287f (127.21.181.67:41115)
I20260812 07:09:56.999172 28898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:37859
I20260812 07:09:57.009809 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:09:57.031013 29281 heartbeater.cc:503] Master 127.21.181.126:43479 was elected leader, sending a full tablet report...
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[28827](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:09:57.060714 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.046394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35446 (local address 127.21.181.65:41913)
0812 07:09:57.047118 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.047125 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.047150 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:09:57.047204 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.047210 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.047431 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.047580 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.049016 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.049995 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.051257 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.051878 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.051989 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.055542 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.055553 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.055568 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.055599 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.058076 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.058698 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.058705 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.058708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.058835 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.059191 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.059194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.059196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.059530 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.059827 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.060144 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.060399 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":370,"threads_started":1}
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:57.076854 29284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.064715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38246 (local address 127.21.181.126:43479)
0812 07:09:57.064913 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.064919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.065057 (+ 138us) server_negotiation.cc:407] Connection header received
0812 07:09:57.065208 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.065214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.065280 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.065360 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.066534 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.067241 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.068478 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.068897 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.069010 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.072517 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.072522 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.072525 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.072556 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.074778 (+ 2222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.075327 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.075330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.075332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.075445 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.075701 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.075704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.075706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.075989 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.076200 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.076410 (+ 210us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.076573 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 07:09:57.077422 28898 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38246
I20260812 07:09:57.078370 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29006
I20260812 07:09:57.089381 29781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.079106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35452 (local address 127.21.181.65:41913)
0812 07:09:57.079248 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.079253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.079556 (+ 303us) server_negotiation.cc:407] Connection header received
0812 07:09:57.079667 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.079670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.079753 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.079827 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.080908 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.081573 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.082697 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.083069 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.083152 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.085109 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.085114 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.085117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.085147 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.087504 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.087973 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.087976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.087978 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.088133 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.088421 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.088424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.088426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.088660 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.088828 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.089004 (+ 176us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.089160 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:09:57.089892 29270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:57.171145 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29006
I20260812 07:09:57.195822 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29290
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:09:57.210656 29792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.196834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.21.181.66:45399)
0812 07:09:57.197374 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.197382 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.197408 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:09:57.197470 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.197478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.197708 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.197839 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.198913 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.199917 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.201094 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.201645 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.201742 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.205651 (+ 3909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.205663 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.205678 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.205711 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.208312 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.208884 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.208890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.208893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.209010 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.209283 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.209286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.209288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.209641 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.209882 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.210096 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.210288 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":267,"threads_started":1}
I20260812 07:09:57.211882 29506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:57.292342 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29290
I20260812 07:09:57.315472 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29517
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[28827](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:09:57.331619 29795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:57.316467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44376 (local address 127.21.181.67:41115)
0812 07:09:57.317112 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:57.317119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:57.317147 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:09:57.317206 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:57.317212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:57.317417 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:57.317563 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:57.319059 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.320198 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.321485 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:57.322057 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:57.322170 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:57.326309 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:57.326321 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:57.326335 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:57.326374 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:57.328941 (+ 2567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.329602 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.329609 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.329611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.329734 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:57.330076 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:57.330080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:57.330082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:57.330420 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:57.330715 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:57.331050 (+ 335us) server_negotiation.cc:299] Negotiation successful
0812 07:09:57.331309 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":329,"threads_started":1}
I20260812 07:09:57.332845 29759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:57.418257 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29517
I20260812 07:09:57.444327 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28848
I20260812 07:09:57.445444 28955 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:09:57.547111 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28848
2026-08-12T07:09:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3977 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[29802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[29802](info): set up 2 sockets
Aug 12 07:09:57 dist-test-slave-lgr5 krb5kdc[29802](info): commencing operation
krb5kdc: starting...
W20260812 07:09:58.784346 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:09:58 dist-test-slave-lgr5 krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:09:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:09:58.972959 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37033
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:37033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:59.287173 29831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:59.287439 29831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:59.287537 29831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:59.294679 29831 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:09:59.294781 29831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:59.294840 29831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:59.294901 29831 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:09:59.294948 29831 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:09:59.304293 29831 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:37033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:59.305387 29831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:59.306573 29831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:59.317345 29847 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:59.317309 29845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:59.317391 29844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:59.318367 29831 server_base.cc:1034] running on GCE node
I20260812 07:09:59.319087 29831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:59.320351 29831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:59.321550 29831 hybrid_clock.cc:648] HybridClock initialized: now 1786518599321510 us; error 35 us; skew 500 ppm
I20260812 07:09:59.321866 29831 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:59 dist-test-slave-lgr5 krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:59.324676 29831 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:09:59.325766 29831 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:59.325867 29831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:59.326184 29831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:59.329041 29831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "df328d67b03a4c2c970e0442bcdef572"
format_stamp: "Formatted at 2026-08-12 07:09:59 on dist-test-slave-lgr5"
server_key: "4624416b447c1851c83ebb49dde13a24"
server_key_iv: "e874d1e7dd08fb8f9d502f4dad73dfe7"
server_key_version: "encryptionkey@0"
I20260812 07:09:59.329792 29831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "df328d67b03a4c2c970e0442bcdef572"
format_stamp: "Formatted at 2026-08-12 07:09:59 on dist-test-slave-lgr5"
server_key: "4624416b447c1851c83ebb49dde13a24"
server_key_iv: "e874d1e7dd08fb8f9d502f4dad73dfe7"
server_key_version: "encryptionkey@0"
I20260812 07:09:59.334417 29831 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:09:59.337591 29863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:59.338724 29831 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:09:59.338871 29831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "df328d67b03a4c2c970e0442bcdef572"
format_stamp: "Formatted at 2026-08-12 07:09:59 on dist-test-slave-lgr5"
server_key: "4624416b447c1851c83ebb49dde13a24"
server_key_iv: "e874d1e7dd08fb8f9d502f4dad73dfe7"
server_key_version: "encryptionkey@0"
I20260812 07:09:59.339074 29831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:59.367273 29831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:59.370841 29831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:59.371078 29831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:59.387629 29831 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:37033
I20260812 07:09:59.387636 29943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:37033 every 8 connection(s)
I20260812 07:09:59.388447 29831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:09:59.389808 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29831
I20260812 07:09:59.390036 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:09:59.390857 22229 external_mini_cluster.cc:1428] Setting key 6c0e6b416e56327be2149163f7cb100e
I20260812 07:09:59.392663 29944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:09:59.403610 29944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572: Bootstrap starting.
Aug 12 07:09:59 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:59.406896 29944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572: Neither blocks nor log segments found. Creating new log.
I20260812 07:09:59.407999 29944 log.cc:824] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572: Log is configured to *not* fsync() on all Append() calls
I20260812 07:09:59.410545 29944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572: No bootstrap required, opened a new log
I20260812 07:09:59.411820 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:59.392882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46782 (local address 127.21.181.126:37033)
0812 07:09:59.393618 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:59.393629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:59.393668 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:09:59.395220 (+ 1552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:59.395241 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:59.395597 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:59.395932 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:59.397448 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:59.398813 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:59.400044 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:59.400757 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:59.400919 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:59.404893 (+ 3974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:59.404901 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:59.404913 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:59.404949 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:59.408016 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:59.408753 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:59.408762 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:59.408768 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:59.408924 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:59.409349 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:59.409353 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:59.409355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:59.409845 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:59.410126 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:59.410572 (+ 446us) server_negotiation.cc:299] Negotiation successful
0812 07:09:59.410939 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":338,"threads_started":1}
I20260812 07:09:59.418530 29944 raft_consensus.cc:348] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df328d67b03a4c2c970e0442bcdef572" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37033 } }
I20260812 07:09:59.418957 29944 raft_consensus.cc:374] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:09:59.419067 29944 raft_consensus.cc:729] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df328d67b03a4c2c970e0442bcdef572, State: Initialized, Role: FOLLOWER
I20260812 07:09:59.419530 29944 consensus_queue.cc:260] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df328d67b03a4c2c970e0442bcdef572" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37033 } }
I20260812 07:09:59.419728 29944 raft_consensus.cc:388] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:09:59.419818 29944 raft_consensus.cc:482] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:09:59.419938 29944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:09:59.421861 29944 raft_consensus.cc:504] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df328d67b03a4c2c970e0442bcdef572" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37033 } }
I20260812 07:09:59.422276 29944 leader_election.cc:302] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df328d67b03a4c2c970e0442bcdef572; no voters:
I20260812 07:09:59.422750 29944 leader_election.cc:288] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:09:59.422866 29955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:09:59.423440 29955 raft_consensus.cc:686] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [term 1 LEADER]: Becoming Leader. State: Replica: df328d67b03a4c2c970e0442bcdef572, State: Running, Role: LEADER
I20260812 07:09:59.423916 29955 consensus_queue.cc:237] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df328d67b03a4c2c970e0442bcdef572" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37033 } }
I20260812 07:09:59.424608 29944 sys_catalog.cc:564] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:09:59.426530 29958 sys_catalog.cc:455] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df328d67b03a4c2c970e0442bcdef572. Latest consensus state: current_term: 1 leader_uuid: "df328d67b03a4c2c970e0442bcdef572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df328d67b03a4c2c970e0442bcdef572" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37033 } } }
I20260812 07:09:59.426614 29957 sys_catalog.cc:455] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df328d67b03a4c2c970e0442bcdef572" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df328d67b03a4c2c970e0442bcdef572" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37033 } } }
I20260812 07:09:59.426936 29958 sys_catalog.cc:458] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:59.427259 29957 sys_catalog.cc:458] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572 [sys.catalog]: This master's current role is: LEADER
I20260812 07:09:59.428005 29979 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:09:59.432821 29979 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:09:59.440245 29979 catalog_manager.cc:1347] Generated new cluster ID: fbc035cd21114006bbd7a777eedf08c9
I20260812 07:09:59.440356 29979 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:09:59.451687 29979 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:09:59.453398 29979 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:09:59.462493 29979 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572: Generated new TSK 0
I20260812 07:09:59.463341 29979 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:09:59.513324 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37033
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:09:59.823427 30001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:09:59.823686 30001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:09:59.823823 30001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:09:59.830909 30001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:09:59.831027 30001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:09:59.831175 30001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:09:59.840445 30001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:09:59.841595 30001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:09:59.842895 30001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:09:59.854791 30010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:59.854827 30016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:09:59.854753 30014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:09:59.855796 30001 server_base.cc:1034] running on GCE node
I20260812 07:09:59.856410 30001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:09:59.857206 30001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:09:59.858390 30001 hybrid_clock.cc:648] HybridClock initialized: now 1786518599858350 us; error 32 us; skew 500 ppm
I20260812 07:09:59.858673 30001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:09:59 dist-test-slave-lgr5 krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:09:59.861685 30001 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:09:59.862707 30001 fs_manager.cc:359] Metadata directory not provided
I20260812 07:09:59.862809 30001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:09:59.863109 30001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:09:59.865916 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "ce4f12fa0f984e29afd6dbcb87381823"
format_stamp: "Formatted at 2026-08-12 07:09:59 on dist-test-slave-lgr5"
server_key: "6d3caaf69695c3d23a89525915bd9fcf"
server_key_iv: "193f0aa28d8f079c311cac082d8a8683"
server_key_version: "encryptionkey@0"
I20260812 07:09:59.866688 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "ce4f12fa0f984e29afd6dbcb87381823"
format_stamp: "Formatted at 2026-08-12 07:09:59 on dist-test-slave-lgr5"
server_key: "6d3caaf69695c3d23a89525915bd9fcf"
server_key_iv: "193f0aa28d8f079c311cac082d8a8683"
server_key_version: "encryptionkey@0"
I20260812 07:09:59.871452 30001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 07:09:59.874779 30026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:59.875941 30001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:09:59.876128 30001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "ce4f12fa0f984e29afd6dbcb87381823"
format_stamp: "Formatted at 2026-08-12 07:09:59 on dist-test-slave-lgr5"
server_key: "6d3caaf69695c3d23a89525915bd9fcf"
server_key_iv: "193f0aa28d8f079c311cac082d8a8683"
server_key_version: "encryptionkey@0"
I20260812 07:09:59.876325 30001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:09:59.890844 30001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:09:59.894299 30001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:09:59.894536 30001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:09:59.895383 30001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:09:59.896651 30001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:09:59.896744 30001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:59.896844 30001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:09:59.896916 30001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:09:59.926589 30001 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:43263
I20260812 07:09:59.926646 30242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:43263 every 8 connection(s)
I20260812 07:09:59.927338 30001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:09:59.930346 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30001
I20260812 07:09:59.930570 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:09:59.931439 22229 external_mini_cluster.cc:1428] Setting key 471680dcbcbfe9f810a378733f97b5e5
Aug 12 07:09:59 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518599, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:09:59.946678 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:59.929717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:41585 (local address 127.21.181.126:37033)
0812 07:09:59.932413 (+ 2696us) server_negotiation.cc:206] Beginning negotiation
0812 07:09:59.932418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:09:59.932442 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:09:59.933103 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:09:59.933107 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:09:59.933166 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:09:59.933260 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:09:59.934755 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:59.935760 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:59.937264 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:09:59.937801 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:09:59.937922 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:59.941160 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:09:59.941165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:09:59.941169 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:09:59.941204 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:09:59.943834 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:59.944653 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:59.944657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:59.944659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:59.944771 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:09:59.945178 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:09:59.945181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:09:59.945183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:09:59.945424 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:09:59.945536 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:09:59.946253 (+ 717us) server_negotiation.cc:299] Negotiation successful
0812 07:09:59.946414 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2588,"thread_start_us":2492,"threads_started":1}
I20260812 07:09:59.947863 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:09:59.930128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37033 (local address 127.21.181.65:41585)
0812 07:09:59.930913 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0812 07:09:59.930966 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:09:59.932832 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:09:59.933514 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:09:59.933525 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:09:59.934057 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:09:59.934584 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:59.934605 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:59.935898 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:59.935902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:59.937125 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:09:59.937138 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:09:59.937977 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:09:59.937982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:09:59.938151 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:09:59.938758 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:09:59.938779 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:09:59.940940 (+ 2161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:09:59.943957 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:59.943966 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:59.943979 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:59.944553 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:59.944863 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:09:59.944867 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:09:59.944870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:09:59.945091 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:09:59.945548 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:09:59.945555 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:09:59.945904 (+ 349us) client_negotiation.cc:769] Sending connection context
0812 07:09:59.946185 (+ 281us) client_negotiation.cc:240] Negotiation successful
0812 07:09:59.946644 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":322,"threads_started":1}
I20260812 07:09:59.949275 30243 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37033
I20260812 07:09:59.949551 30243 heartbeater.cc:464] Registering TS with master...
I20260812 07:09:59.950157 30243 heartbeater.cc:511] Master 127.21.181.126:37033 requested a full tablet report, sending...
I20260812 07:09:59.952176 29891 ts_manager.cc:194] Registered new tserver with Master: ce4f12fa0f984e29afd6dbcb87381823 (127.21.181.65:43263)
I20260812 07:09:59.953583 29891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:41585
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:09:59.984539 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37033
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:00.306885 30254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:00.307129 30254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:00.307264 30254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:00.314288 30254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:00.314396 30254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:00.314556 30254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:00.323954 30254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:00.325153 30254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:00.326468 30254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:00.338908 30270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:00.338866 30271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:00.338946 30274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:00.340032 30254 server_base.cc:1034] running on GCE node
I20260812 07:10:00.340627 30254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:00.341528 30254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:00.342721 30254 hybrid_clock.cc:648] HybridClock initialized: now 1786518600342656 us; error 60 us; skew 500 ppm
I20260812 07:10:00.343032 30254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:00 dist-test-slave-lgr5 krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:00.349866 30254 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:10:00.350926 30254 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:00.351027 30254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:00.351322 30254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:00.354216 30254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "e01c93ebaed24d489488605324f1e3b5"
format_stamp: "Formatted at 2026-08-12 07:10:00 on dist-test-slave-lgr5"
server_key: "2e6e7dee1fc7451859d1ac39a2240bb4"
server_key_iv: "b02e17f27cfb1a6494f0b36fdcc713ce"
server_key_version: "encryptionkey@0"
I20260812 07:10:00.355011 30254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "e01c93ebaed24d489488605324f1e3b5"
format_stamp: "Formatted at 2026-08-12 07:10:00 on dist-test-slave-lgr5"
server_key: "2e6e7dee1fc7451859d1ac39a2240bb4"
server_key_iv: "b02e17f27cfb1a6494f0b36fdcc713ce"
server_key_version: "encryptionkey@0"
I20260812 07:10:00.359823 30254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 07:10:00.363008 30286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:00.364262 30254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260812 07:10:00.364427 30254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "e01c93ebaed24d489488605324f1e3b5"
format_stamp: "Formatted at 2026-08-12 07:10:00 on dist-test-slave-lgr5"
server_key: "2e6e7dee1fc7451859d1ac39a2240bb4"
server_key_iv: "b02e17f27cfb1a6494f0b36fdcc713ce"
server_key_version: "encryptionkey@0"
I20260812 07:10:00.364634 30254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:00.385596 30254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:00.389514 30254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:00.389750 30254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:00.390627 30254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:00.391930 30254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:00.392014 30254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:00.392143 30254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:00.392207 30254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:00.424635 30254 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:37229
I20260812 07:10:00.424702 30479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:37229 every 8 connection(s)
I20260812 07:10:00.425467 30254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:00.427425 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30254
I20260812 07:10:00.427624 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:00.428238 22229 external_mini_cluster.cc:1428] Setting key 044457c435ed6f3273fb8613880e219e
Aug 12 07:10:00 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:00.445226 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.427874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:55261 (local address 127.21.181.126:37033)
0812 07:10:00.428049 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:00.428054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:00.429516 (+ 1462us) server_negotiation.cc:407] Connection header received
0812 07:10:00.431620 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:00.431625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:00.431706 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:00.431809 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:00.433238 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.434152 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.435684 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.436280 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.436409 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.439654 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:00.439659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:00.439662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:00.439693 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:00.442211 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:00.442901 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:00.442907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:00.442911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:00.443046 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:00.443544 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:00.443549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:00.443553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:00.443818 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:00.443973 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:00.444671 (+ 698us) server_negotiation.cc:299] Negotiation successful
0812 07:10:00.444905 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:10:00.446159 30487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.428496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37033 (local address 127.21.181.66:55261)
0812 07:10:00.429344 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:00.429399 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:00.431347 (+ 1948us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:00.431978 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:00.431986 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:00.432532 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:00.433071 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.433089 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.434324 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.434328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.435543 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.435555 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.436464 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.436468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.436653 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.437261 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:00.437283 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:00.439451 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:00.442336 (+ 2885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:00.442346 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:00.442361 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:00.442782 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:00.443196 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:00.443200 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:00.443202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:00.443443 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:00.443945 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:00.443953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:00.444321 (+ 368us) client_negotiation.cc:769] Sending connection context
0812 07:10:00.444569 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 07:10:00.445065 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260812 07:10:00.447563 30481 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37033
I20260812 07:10:00.447840 30481 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:00.448490 30481 heartbeater.cc:511] Master 127.21.181.126:37033 requested a full tablet report, sending...
I20260812 07:10:00.449833 29891 ts_manager.cc:194] Registered new tserver with Master: e01c93ebaed24d489488605324f1e3b5 (127.21.181.66:37229)
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
I20260812 07:10:00.450718 29891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:55261
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:10:00.475111 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37033
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:45445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:00.783622 30491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:00.783864 30491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:00.783960 30491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:00.790745 30491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:00.790834 30491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:00.790946 30491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:00.800072 30491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:45445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:00.801195 30491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:00.802372 30491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:00.814047 30500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:00.813974 30502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:00.814078 30505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:00.814755 30491 server_base.cc:1034] running on GCE node
I20260812 07:10:00.815310 30491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:00.816154 30491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:00.817382 30491 hybrid_clock.cc:648] HybridClock initialized: now 1786518600817333 us; error 36 us; skew 500 ppm
I20260812 07:10:00.817696 30491 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:00 dist-test-slave-lgr5 krb5kdc[29802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:00.820607 30491 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:10:00.821601 30491 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:00.821686 30491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:00.821936 30491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:00.824677 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "78c28f685e7f466fb194ad5107b0b60e"
format_stamp: "Formatted at 2026-08-12 07:10:00 on dist-test-slave-lgr5"
server_key: "795c1a544012431d2ec81f73d6537ca0"
server_key_iv: "3a2c9b468a6fb7733f0559da71a26526"
server_key_version: "encryptionkey@0"
I20260812 07:10:00.825454 30491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "78c28f685e7f466fb194ad5107b0b60e"
format_stamp: "Formatted at 2026-08-12 07:10:00 on dist-test-slave-lgr5"
server_key: "795c1a544012431d2ec81f73d6537ca0"
server_key_iv: "3a2c9b468a6fb7733f0559da71a26526"
server_key_version: "encryptionkey@0"
I20260812 07:10:00.830217 30491 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:10:00.833498 30515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:00.834664 30491 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:10:00.834805 30491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "78c28f685e7f466fb194ad5107b0b60e"
format_stamp: "Formatted at 2026-08-12 07:10:00 on dist-test-slave-lgr5"
server_key: "795c1a544012431d2ec81f73d6537ca0"
server_key_iv: "3a2c9b468a6fb7733f0559da71a26526"
server_key_version: "encryptionkey@0"
I20260812 07:10:00.834988 30491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:00.855167 30491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:00.859004 30491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:00.859210 30491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:00.860044 30491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:00.861307 30491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:00.861392 30491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:00.861469 30491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:00.861531 30491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:00.889457 30491 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:46433
I20260812 07:10:00.889511 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:46433 every 8 connection(s)
I20260812 07:10:00.890232 30491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:00.891621 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30491
I20260812 07:10:00.891793 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:00.892411 22229 external_mini_cluster.cc:1428] Setting key 5376307e6a38693704e23559fc79568a
Aug 12 07:10:00 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518600, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:00.908739 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.892589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:36585 (local address 127.21.181.126:37033)
0812 07:10:00.892752 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:00.892757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:00.894243 (+ 1486us) server_negotiation.cc:407] Connection header received
0812 07:10:00.896037 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:00.896041 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:00.896139 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:00.896214 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:00.897598 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.898314 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.899749 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.900193 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.900279 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.903490 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:00.903504 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:00.903507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:00.903540 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:00.905855 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:00.906644 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:00.906649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:00.906652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:00.906793 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:00.907221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:00.907225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:00.907229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:00.907579 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:00.907687 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:00.908328 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 07:10:00.908479 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:10:00.909642 30735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.893255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37033 (local address 127.21.181.67:36585)
0812 07:10:00.894069 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:00.894123 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:00.895804 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:00.896421 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:00.896428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:00.896915 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:00.897451 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.897469 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.898433 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.898437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.899645 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.899657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.900401 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.900406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.900590 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.901204 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:00.901226 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:00.903276 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:00.905985 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:00.905993 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:00.906007 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:00.906540 (+ 533us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:00.906935 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:00.906938 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:00.906941 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:00.907120 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:00.907710 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:00.907715 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:00.907992 (+ 277us) client_negotiation.cc:769] Sending connection context
0812 07:10:00.908249 (+ 257us) client_negotiation.cc:240] Negotiation successful
0812 07:10:00.908605 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":343,"threads_started":1}
I20260812 07:10:00.910712 30732 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37033
I20260812 07:10:00.910977 30732 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:00.911580 30732 heartbeater.cc:511] Master 127.21.181.126:37033 requested a full tablet report, sending...
I20260812 07:10:00.912878 29891 ts_manager.cc:194] Registered new tserver with Master: 78c28f685e7f466fb194ad5107b0b60e (127.21.181.67:46433)
I20260812 07:10:00.913619 29891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:36585
I20260812 07:10:00.919318 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:10:00.935892 30248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.923588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.21.181.126:37033)
0812 07:10:00.923726 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:00.923730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:00.924252 (+ 522us) server_negotiation.cc:407] Connection header received
0812 07:10:00.924385 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:00.924388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:00.924440 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:00.924517 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:00.925856 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.926557 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.928349 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.928807 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.928890 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.931540 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:00.931545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:00.931548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:00.931576 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:00.933883 (+ 2307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:00.934454 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:00.934458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:00.934460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:00.934585 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:00.934850 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:00.934853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:00.934855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:00.935146 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:00.935326 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:00.935537 (+ 211us) server_negotiation.cc:299] Negotiation successful
0812 07:10:00.935681 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:10:00.939306 29891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:00.945973 29891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:00.956426 30243 heartbeater.cc:503] Master 127.21.181.126:37033 was elected leader, sending a full tablet report...
I20260812 07:10:00.976497 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.956479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:46433 (local address 127.0.0.1:51290)
0812 07:10:00.968106 (+ 11627us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:00.968145 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:00.969016 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:00.970077 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:00.970084 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:00.970113 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:00.970594 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.970606 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.972204 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.972209 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.974147 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.974159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.975511 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.975517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.975709 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.975758 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 07:10:00.975892 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 07:10:00.976062 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11380,"mutex_wait_us":3,"spinlock_wait_cycles":12672,"thread_start_us":11923,"threads_started":1}
I20260812 07:10:00.977188 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.956614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.21.181.67:46433)
0812 07:10:00.957815 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:00.957823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:00.968240 (+ 10417us) server_negotiation.cc:407] Connection header received
0812 07:10:00.969257 (+ 1017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:00.969265 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:00.969494 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:00.969663 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:00.970840 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.972009 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.974296 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.975319 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.975443 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.975671 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:00.975949 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:10:00.976135 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":890,"threads_started":1}
I20260812 07:10:00.977771 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.956728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:37229 (local address 127.0.0.1:37480)
0812 07:10:00.968980 (+ 12252us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:00.968997 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:00.969106 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:00.970208 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:00.970213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:00.970243 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:00.970641 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.970650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.972609 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.972614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.974880 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.974895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.976884 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.976891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.977081 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.977111 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:10:00.977185 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 07:10:00.977306 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12119,"mutex_wait_us":3,"thread_start_us":10997,"threads_started":1}
I20260812 07:10:00.978206 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.957272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37480 (local address 127.21.181.66:37229)
0812 07:10:00.968844 (+ 11572us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:00.968852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:00.969229 (+ 377us) server_negotiation.cc:407] Connection header received
0812 07:10:00.969318 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:00.969324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:00.969509 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:00.969664 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:00.970823 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.972451 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.975072 (+ 2621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.976377 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.976511 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.977656 (+ 1145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:00.977728 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 07:10:00.977886 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11211,"thread_start_us":11330,"threads_started":1}
I20260812 07:10:00.978338 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.958286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40054 (local address 127.21.181.65:43263)
0812 07:10:00.970083 (+ 11797us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:00.970091 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:00.970117 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:10:00.970347 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:00.970355 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:00.970582 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:00.970770 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:00.971333 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.972806 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.975219 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.976556 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.976690 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.976906 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:00.977883 (+ 977us) server_negotiation.cc:299] Negotiation successful
0812 07:10:00.978041 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11636,"thread_start_us":11498,"threads_started":1}
I20260812 07:10:00.977795 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:00.957939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43263 (local address 127.0.0.1:40054)
0812 07:10:00.970041 (+ 12102us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:00.970056 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:00.970146 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:00.970767 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:00.970772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:00.970795 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:00.971135 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.971146 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.973000 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.973006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.975080 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:00.975091 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:00.977033 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:00.977039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:00.977198 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:00.977231 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:10:00.977319 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:10:00.977441 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11967,"thread_start_us":10787,"threads_started":1}
I20260812 07:10:00.980782 30597 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa5d05163264ac1850fd9ff4f226461 (DEFAULT_TABLE table=test-table [id=e5ce141f3b3241ca8987f41d13458614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:00.981611 30366 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa5d05163264ac1850fd9ff4f226461 (DEFAULT_TABLE table=test-table [id=e5ce141f3b3241ca8987f41d13458614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:00.982235 30597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa5d05163264ac1850fd9ff4f226461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:00.981850 30121 tablet_service.cc:1467] Processing CreateTablet for tablet 8fa5d05163264ac1850fd9ff4f226461 (DEFAULT_TABLE table=test-table [id=e5ce141f3b3241ca8987f41d13458614]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:00.982930 30366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa5d05163264ac1850fd9ff4f226461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:00.983023 30121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8fa5d05163264ac1850fd9ff4f226461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:00.992749 30760 tablet_bootstrap.cc:492] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Bootstrap starting.
I20260812 07:10:00.996009 30760 tablet_bootstrap.cc:654] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:00.996817 30763 tablet_bootstrap.cc:492] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Bootstrap starting.
I20260812 07:10:00.997624 30762 tablet_bootstrap.cc:492] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Bootstrap starting.
I20260812 07:10:00.997633 30760 log.cc:824] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:01.000206 30763 tablet_bootstrap.cc:654] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:01.001153 30762 tablet_bootstrap.cc:654] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:01.001329 30763 log.cc:824] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:01.002334 30762 log.cc:824] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:01.002840 30760 tablet_bootstrap.cc:492] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: No bootstrap required, opened a new log
I20260812 07:10:01.003111 30760 ts_tablet_manager.cc:1397] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.005s
I20260812 07:10:01.003907 30763 tablet_bootstrap.cc:492] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: No bootstrap required, opened a new log
I20260812 07:10:01.004177 30763 ts_tablet_manager.cc:1397] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260812 07:10:01.011284 30762 tablet_bootstrap.cc:492] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: No bootstrap required, opened a new log
I20260812 07:10:01.011147 30763 raft_consensus.cc:348] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.011493 30763 raft_consensus.cc:374] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:01.011534 30762 ts_tablet_manager.cc:1397] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260812 07:10:01.011595 30763 raft_consensus.cc:729] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce4f12fa0f984e29afd6dbcb87381823, State: Initialized, Role: FOLLOWER
I20260812 07:10:01.012187 30763 consensus_queue.cc:260] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.013854 30763 ts_tablet_manager.cc:1428] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 07:10:01.013221 30760 raft_consensus.cc:348] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.014246 30760 raft_consensus.cc:374] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:01.014343 30760 raft_consensus.cc:729] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78c28f685e7f466fb194ad5107b0b60e, State: Initialized, Role: FOLLOWER
I20260812 07:10:01.014873 30760 consensus_queue.cc:260] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.018348 30760 ts_tablet_manager.cc:1428] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260812 07:10:01.018492 30732 heartbeater.cc:503] Master 127.21.181.126:37033 was elected leader, sending a full tablet report...
I20260812 07:10:01.021173 30762 raft_consensus.cc:348] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.021485 30762 raft_consensus.cc:374] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:01.021576 30762 raft_consensus.cc:729] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e01c93ebaed24d489488605324f1e3b5, State: Initialized, Role: FOLLOWER
I20260812 07:10:01.022122 30762 consensus_queue.cc:260] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.023031 30481 heartbeater.cc:503] Master 127.21.181.126:37033 was elected leader, sending a full tablet report...
I20260812 07:10:01.023365 30762 ts_tablet_manager.cc:1428] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260812 07:10:01.141573 30734 tablet.cc:2369] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:10:01.177160 30486 tablet.cc:2369] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:10:01.179414 30245 tablet.cc:2369] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:01.364106 30773 raft_consensus.cc:482] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:01.364442 30773 raft_consensus.cc:504] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.366175 30773 leader_election.cc:288] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78c28f685e7f466fb194ad5107b0b60e (127.21.181.67:46433), ce4f12fa0f984e29afd6dbcb87381823 (127.21.181.65:43263)
I20260812 07:10:01.374408 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.366558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:48811 (local address 127.21.181.67:46433)
0812 07:10:01.366757 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:01.366764 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:01.367417 (+ 653us) server_negotiation.cc:407] Connection header received
0812 07:10:01.367657 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:01.367663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:01.367723 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:01.367796 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:01.368416 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.369553 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.372264 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.373442 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.373604 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.373674 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:01.374038 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 07:10:01.374165 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:10:01.374411 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.366849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:53125 (local address 127.21.181.65:43263)
0812 07:10:01.367052 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:01.367058 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:01.367676 (+ 618us) server_negotiation.cc:407] Connection header received
0812 07:10:01.367844 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:01.367848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:01.367906 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:01.367991 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:01.368619 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.369673 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.372255 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.373438 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.373600 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.373674 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:01.374042 (+ 368us) server_negotiation.cc:299] Negotiation successful
0812 07:10:01.374164 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:10:01.374615 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.366655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43263 (local address 127.21.181.66:53125)
0812 07:10:01.367559 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:01.367578 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:01.367712 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:01.368021 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:01.368029 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:01.368054 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:01.368459 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:01.368469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.369805 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.369810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:01.372102 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:01.372118 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.373686 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.373692 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:01.373909 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.373935 (+ 26us) client_negotiation.cc:769] Sending connection context
0812 07:10:01.374021 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:10:01.374180 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":787,"mutex_wait_us":157,"thread_start_us":577,"threads_started":1}
I20260812 07:10:01.374943 30774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.366433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:46433 (local address 127.21.181.66:48811)
0812 07:10:01.367284 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:01.367302 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:01.367504 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:01.367826 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:01.367831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:01.367856 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:01.368269 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:01.368283 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.369669 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.369672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:01.372103 (+ 2431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:01.372118 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.373667 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.373678 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:01.373865 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.373914 (+ 49us) client_negotiation.cc:769] Sending connection context
0812 07:10:01.374013 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:10:01.374180 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"mutex_wait_us":84,"thread_start_us":518,"threads_started":1}
I20260812 07:10:01.375703 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa5d05163264ac1850fd9ff4f226461" candidate_uuid: "e01c93ebaed24d489488605324f1e3b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4f12fa0f984e29afd6dbcb87381823" is_pre_election: true
I20260812 07:10:01.375867 30637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa5d05163264ac1850fd9ff4f226461" candidate_uuid: "e01c93ebaed24d489488605324f1e3b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78c28f685e7f466fb194ad5107b0b60e" is_pre_election: true
I20260812 07:10:01.376332 30161 raft_consensus.cc:2454] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01c93ebaed24d489488605324f1e3b5 in term 0.
I20260812 07:10:01.376332 30637 raft_consensus.cc:2454] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e01c93ebaed24d489488605324f1e3b5 in term 0.
I20260812 07:10:01.376971 30292 leader_election.cc:302] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce4f12fa0f984e29afd6dbcb87381823, e01c93ebaed24d489488605324f1e3b5; no voters:
I20260812 07:10:01.377295 30773 raft_consensus.cc:2781] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:01.377439 30773 raft_consensus.cc:482] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:01.377547 30773 raft_consensus.cc:3037] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:01.379668 30773 raft_consensus.cc:504] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.380281 30773 leader_election.cc:288] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [CANDIDATE]: Term 1 election: Requested vote from peers 78c28f685e7f466fb194ad5107b0b60e (127.21.181.67:46433), ce4f12fa0f984e29afd6dbcb87381823 (127.21.181.65:43263)
I20260812 07:10:01.380622 30637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa5d05163264ac1850fd9ff4f226461" candidate_uuid: "e01c93ebaed24d489488605324f1e3b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78c28f685e7f466fb194ad5107b0b60e"
I20260812 07:10:01.380740 30161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8fa5d05163264ac1850fd9ff4f226461" candidate_uuid: "e01c93ebaed24d489488605324f1e3b5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce4f12fa0f984e29afd6dbcb87381823"
I20260812 07:10:01.380800 30637 raft_consensus.cc:3037] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:01.380939 30161 raft_consensus.cc:3037] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:01.382913 30637 raft_consensus.cc:2454] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01c93ebaed24d489488605324f1e3b5 in term 1.
I20260812 07:10:01.383037 30161 raft_consensus.cc:2454] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e01c93ebaed24d489488605324f1e3b5 in term 1.
I20260812 07:10:01.383291 30295 leader_election.cc:302] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78c28f685e7f466fb194ad5107b0b60e, e01c93ebaed24d489488605324f1e3b5; no voters:
I20260812 07:10:01.383630 30773 raft_consensus.cc:2781] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:01.384052 30773 raft_consensus.cc:686] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 1 LEADER]: Becoming Leader. State: Replica: e01c93ebaed24d489488605324f1e3b5, State: Running, Role: LEADER
I20260812 07:10:01.384528 30773 consensus_queue.cc:237] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } }
I20260812 07:10:01.389297 29890 catalog_manager.cc:5495] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 reported cstate change: term changed from 0 to 1, leader changed from <none> to e01c93ebaed24d489488605324f1e3b5 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "e01c93ebaed24d489488605324f1e3b5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e01c93ebaed24d489488605324f1e3b5" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37229 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 } health_report { overall_health: UNKNOWN } } }
I20260812 07:10:01.423708 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.418939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37488 (local address 127.21.181.66:37229)
0812 07:10:01.419111 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:01.419116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:01.419273 (+ 157us) server_negotiation.cc:407] Connection header received
0812 07:10:01.419432 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:01.419435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:01.419521 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:01.419640 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:01.419914 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.420662 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.421982 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.422367 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.422457 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.422825 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:01.423310 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:01.423413 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:01.423452 (+ 39us) server_negotiation.cc:299] Negotiation successful
0812 07:10:01.423517 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:10:01.431707 30637 raft_consensus.cc:1261] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 1 FOLLOWER]: Refusing update from remote peer e01c93ebaed24d489488605324f1e3b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:01.431768 30161 raft_consensus.cc:1261] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 1 FOLLOWER]: Refusing update from remote peer e01c93ebaed24d489488605324f1e3b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:01.432715 30778 consensus_queue.cc:1035] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce4f12fa0f984e29afd6dbcb87381823" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:01.433071 30773 consensus_queue.cc:1035] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78c28f685e7f466fb194ad5107b0b60e" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:01.444792 30783 mvcc.cc:205] Tried to move back new op lower bound from 7317580191454601216 to 7317580191274557440. Current Snapshot: MvccSnapshot[applied={T|T < 7317580191454601216}]
I20260812 07:10:01.445550 30785 mvcc.cc:205] Tried to move back new op lower bound from 7317580191454601216 to 7317580191274557440. Current Snapshot: MvccSnapshot[applied={T|T < 7317580191454601216}]
I20260812 07:10:01.450599 30784 mvcc.cc:205] Tried to move back new op lower bound from 7317580191454601216 to 7317580191274557440. Current Snapshot: MvccSnapshot[applied={T|T < 7317580191454601216}]
I20260812 07:10:01.457067 29891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:01.457340 29891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:01.460695 29891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df328d67b03a4c2c970e0442bcdef572: Sending DeleteTablet for 3 replicas of tablet 8fa5d05163264ac1850fd9ff4f226461
I20260812 07:10:01.461652 30597 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fa5d05163264ac1850fd9ff4f226461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:01 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:51290
I20260812 07:10:01.461732 30121 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fa5d05163264ac1850fd9ff4f226461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:01 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:40054
I20260812 07:10:01.461746 30366 tablet_service.cc:1514] Processing DeleteTablet for tablet 8fa5d05163264ac1850fd9ff4f226461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:01 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:37480
I20260812 07:10:01.463913 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30001
I20260812 07:10:01.474872 30799 tablet_replica.cc:330] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: stopping tablet replica
I20260812 07:10:01.475343 30799 raft_consensus.cc:2229] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:01.475848 30799 raft_consensus.cc:2258] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:01.478447 30799 ts_tablet_manager.cc:1905] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:01.485491 30799 ts_tablet_manager.cc:1918] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:01.485661 30799 log.cc:1196] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/8fa5d05163264ac1850fd9ff4f226461
I20260812 07:10:01.486186 30799 ts_tablet_manager.cc:1939] T 8fa5d05163264ac1850fd9ff4f226461 P ce4f12fa0f984e29afd6dbcb87381823: Deleting consensus metadata
I20260812 07:10:01.488014 29865 catalog_manager.cc:4841] TS ce4f12fa0f984e29afd6dbcb87381823 (127.21.181.65:43263): tablet 8fa5d05163264ac1850fd9ff4f226461 (table test-table [id=e5ce141f3b3241ca8987f41d13458614]) successfully deleted
I20260812 07:10:01.488903 30798 tablet_replica.cc:330] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: stopping tablet replica
I20260812 07:10:01.489379 30798 raft_consensus.cc:2229] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:01.489871 30798 raft_consensus.cc:2258] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:01.490697 30801 tablet_replica.cc:330] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: stopping tablet replica
I20260812 07:10:01.491120 30801 raft_consensus.cc:2229] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:01.491772 30801 raft_consensus.cc:2258] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:01.492842 30798 ts_tablet_manager.cc:1905] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:01.493518 30801 ts_tablet_manager.cc:1905] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 07:10:01 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:10:01.499665 30798 ts_tablet_manager.cc:1918] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:01.499894 30798 log.cc:1196] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/8fa5d05163264ac1850fd9ff4f226461
I20260812 07:10:01.500492 30798 ts_tablet_manager.cc:1939] T 8fa5d05163264ac1850fd9ff4f226461 P 78c28f685e7f466fb194ad5107b0b60e: Deleting consensus metadata
I20260812 07:10:01.500490 30801 ts_tablet_manager.cc:1918] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:01.500643 30801 log.cc:1196] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/8fa5d05163264ac1850fd9ff4f226461
I20260812 07:10:01.501204 30801 ts_tablet_manager.cc:1939] T 8fa5d05163264ac1850fd9ff4f226461 P e01c93ebaed24d489488605324f1e3b5: Deleting consensus metadata
I20260812 07:10:01.502454 29872 catalog_manager.cc:4841] TS 78c28f685e7f466fb194ad5107b0b60e (127.21.181.67:46433): tablet 8fa5d05163264ac1850fd9ff4f226461 (table test-table [id=e5ce141f3b3241ca8987f41d13458614]) successfully deleted
I20260812 07:10:01.502792 29864 catalog_manager.cc:4841] TS e01c93ebaed24d489488605324f1e3b5 (127.21.181.66:37229): tablet 8fa5d05163264ac1850fd9ff4f226461 (table test-table [id=e5ce141f3b3241ca8987f41d13458614]) successfully deleted
I20260812 07:10:01.507521 30758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.465639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.21.181.65:43263)
0812 07:10:01.465816 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:01.465824 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:01.487975 (+ 22151us) server_negotiation.cc:407] Connection header received
0812 07:10:01.488600 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:01.488605 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:01.488695 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:01.488770 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:01.490890 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.492003 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.494358 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.494964 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.495091 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.500715 (+ 5624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:01.500731 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:01.500756 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:01.500804 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:01.504612 (+ 3808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:01.505435 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:01.505445 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:01.505449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:01.505649 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:01.505968 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:01.505973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:01.505976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:01.506448 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:01.506766 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:01.506923 (+ 157us) server_negotiation.cc:299] Negotiation successful
0812 07:10:01.507139 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:10:01.508204 30238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:01.596841 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30001
I20260812 07:10:01.622900 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30254
Aug 12 07:10:01 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:10:01.639658 30755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.623749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37494 (local address 127.21.181.66:37229)
0812 07:10:01.623952 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:01.623959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:01.623982 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:10:01.624101 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:01.624107 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:01.624201 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:01.624317 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:01.625553 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.626799 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.627977 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.628840 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.629005 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.633145 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:01.633161 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:01.633181 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:01.633226 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:01.636997 (+ 3771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:01.637589 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:01.637598 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:01.637602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:01.637751 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:01.638056 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:01.638061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:01.638064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:01.638555 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:01.638888 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:01.639044 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 07:10:01.639258 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:10:01.640368 30469 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:01.749497 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30254
I20260812 07:10:01.774711 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30491
Aug 12 07:10:01 dist-test-slave-lgr5 krb5kdc[29802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:10:01.788259 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:01.775443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51296 (local address 127.21.181.67:46433)
0812 07:10:01.775652 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:01.775657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:01.775676 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:10:01.775795 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:01.775799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:01.775874 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:01.775953 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:01.777177 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.777873 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.778998 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:01.779372 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:01.779487 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:01.783577 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:01.783591 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:01.783607 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:01.783640 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:01.786073 (+ 2433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:01.786632 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:01.786639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:01.786641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:01.786758 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:01.787046 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:01.787048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:01.787050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:01.787386 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:01.787607 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:01.787796 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 07:10:01.787954 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260812 07:10:01.788905 30723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:01.874126 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30491
I20260812 07:10:01.898415 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29831
I20260812 07:10:01.899559 29934 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:02.008823 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29831
2026-08-12T07:10:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4456 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-12T07:10:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:02Z Disabled control of system clock
I20260812 07:10:02.081688 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39941
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:39941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 07:10:02.393923 30815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:02.394186 30815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:02.394273 30815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:02.401276 30815 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:02.401378 30815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:02.401427 30815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:02.401481 30815 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:02.401522 30815 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:02.410432 30815 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:39941
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:02.411586 30815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:02.413309 30815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:02.425449 30831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:02.425593 30827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:02.425570 30833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:02.426365 30815 server_base.cc:1034] running on GCE node
I20260812 07:10:02.427016 30815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:02.428296 30815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:02.429503 30815 hybrid_clock.cc:648] HybridClock initialized: now 1786518602429453 us; error 40 us; skew 500 ppm
I20260812 07:10:02.429801 30815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:02.431905 30815 webserver.cc:460] Webserver started at http://127.21.181.126:34233/ using document root <none> and password file <none>
I20260812 07:10:02.432565 30815 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:02.432649 30815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:02.432910 30815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:02.435608 30815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "33524a6a249b441490bbe2af75ae9d91"
format_stamp: "Formatted at 2026-08-12 07:10:02 on dist-test-slave-lgr5"
server_key: "24d84bd815ac9fdd5c00f261ff8ae9c3"
server_key_iv: "0d4fdaec09e8843b4d59f8c900dad08a"
server_key_version: "encryptionkey@0"
I20260812 07:10:02.436383 30815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "33524a6a249b441490bbe2af75ae9d91"
format_stamp: "Formatted at 2026-08-12 07:10:02 on dist-test-slave-lgr5"
server_key: "24d84bd815ac9fdd5c00f261ff8ae9c3"
server_key_iv: "0d4fdaec09e8843b4d59f8c900dad08a"
server_key_version: "encryptionkey@0"
I20260812 07:10:02.440945 30815 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:10:02.443998 30847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:02.445187 30815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:10:02.445351 30815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "33524a6a249b441490bbe2af75ae9d91"
format_stamp: "Formatted at 2026-08-12 07:10:02 on dist-test-slave-lgr5"
server_key: "24d84bd815ac9fdd5c00f261ff8ae9c3"
server_key_iv: "0d4fdaec09e8843b4d59f8c900dad08a"
server_key_version: "encryptionkey@0"
I20260812 07:10:02.445536 30815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:02.450824 30815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:02.451514 30815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:02.451778 30815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:02.468350 30815 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:39941
I20260812 07:10:02.468374 30943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:39941 every 8 connection(s)
I20260812 07:10:02.469524 30815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:02.472911 30945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:02.477788 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30815
I20260812 07:10:02.477958 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:02.478585 22229 external_mini_cluster.cc:1428] Setting key 0ef261f23f86b5f7762ad84bd5a0c3e9
I20260812 07:10:02.483291 30945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91: Bootstrap starting.
I20260812 07:10:02.486407 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:02.480522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44414 (local address 127.21.181.126:39941)
0812 07:10:02.481544 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:02.481555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:02.481597 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:10:02.483152 (+ 1555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:02.483174 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:02.483574 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:02.483875 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:02.484968 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:02.484974 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:02.484985 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:02.485000 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:02.485062 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:02.485080 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:02.485237 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:02.485340 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 07:10:02.485569 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":376,"threads_started":1}
I20260812 07:10:02.486732 30945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:02.487964 30945 log.cc:824] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:02.490700 30945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91: No bootstrap required, opened a new log
I20260812 07:10:02.497553 30945 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33524a6a249b441490bbe2af75ae9d91" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39941 } }
I20260812 07:10:02.497859 30945 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:02.497956 30945 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33524a6a249b441490bbe2af75ae9d91, State: Initialized, Role: FOLLOWER
I20260812 07:10:02.498548 30945 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33524a6a249b441490bbe2af75ae9d91" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39941 } }
I20260812 07:10:02.498764 30945 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:02.498890 30945 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:02.499042 30945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:02.501096 30945 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33524a6a249b441490bbe2af75ae9d91" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39941 } }
I20260812 07:10:02.501542 30945 leader_election.cc:302] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33524a6a249b441490bbe2af75ae9d91; no voters:
I20260812 07:10:02.502058 30945 leader_election.cc:288] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:02.502178 30950 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:02.502804 30950 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [term 1 LEADER]: Becoming Leader. State: Replica: 33524a6a249b441490bbe2af75ae9d91, State: Running, Role: LEADER
I20260812 07:10:02.503296 30950 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33524a6a249b441490bbe2af75ae9d91" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39941 } }
I20260812 07:10:02.503899 30945 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:02.505786 30951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33524a6a249b441490bbe2af75ae9d91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33524a6a249b441490bbe2af75ae9d91" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39941 } } }
I20260812 07:10:02.506181 30951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:02.506064 30952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33524a6a249b441490bbe2af75ae9d91. Latest consensus state: current_term: 1 leader_uuid: "33524a6a249b441490bbe2af75ae9d91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33524a6a249b441490bbe2af75ae9d91" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39941 } } }
I20260812 07:10:02.506366 30952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:02.507983 30958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:02.512422 30958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:02.521436 30958 catalog_manager.cc:1347] Generated new cluster ID: 84a1f52d538547b49a25ad5c38048cc6
I20260812 07:10:02.521524 30958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:02.540105 30958 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:02.540457 30958 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:02.554111 30958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91: Generated new TSK 0
I20260812 07:10:02.554791 30958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:10:02.560241 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39941
--builtin_ntp_servers=127.21.181.84:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260812 07:10:02.890715 30978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:02.890967 30978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:02.891067 30978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:02.898039 30978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:02.898125 30978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:02.898233 30978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:02.906919 30978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:02.908003 30978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:02.909684 30978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:02.921854 30988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:02.921905 30989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:02.921877 30992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:02.922910 30978 server_base.cc:1034] running on GCE node
I20260812 07:10:02.923426 30978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:02.924296 30978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:02.925488 30978 hybrid_clock.cc:648] HybridClock initialized: now 1786518602925436 us; error 37 us; skew 500 ppm
I20260812 07:10:02.925746 30978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:02.927788 30978 webserver.cc:460] Webserver started at http://127.21.181.65:37249/ using document root <none> and password file <none>
I20260812 07:10:02.928438 30978 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:02.928534 30978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:02.928802 30978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:02.931349 30978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "101e86557f984b519580eb046c606edd"
format_stamp: "Formatted at 2026-08-12 07:10:02 on dist-test-slave-lgr5"
server_key: "eae83ac8e06cfa4b1cfe049167051346"
server_key_iv: "83f63237e4c0aa6f64fdc94bf0afba54"
server_key_version: "encryptionkey@0"
I20260812 07:10:02.932062 30978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "101e86557f984b519580eb046c606edd"
format_stamp: "Formatted at 2026-08-12 07:10:02 on dist-test-slave-lgr5"
server_key: "eae83ac8e06cfa4b1cfe049167051346"
server_key_iv: "83f63237e4c0aa6f64fdc94bf0afba54"
server_key_version: "encryptionkey@0"
I20260812 07:10:02.936780 30978 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:10:02.939908 31002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:02.941092 30978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:10:02.941224 30978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "101e86557f984b519580eb046c606edd"
format_stamp: "Formatted at 2026-08-12 07:10:02 on dist-test-slave-lgr5"
server_key: "eae83ac8e06cfa4b1cfe049167051346"
server_key_iv: "83f63237e4c0aa6f64fdc94bf0afba54"
server_key_version: "encryptionkey@0"
I20260812 07:10:02.941413 30978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:02.946743 30978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:02.947424 30978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:02.947664 30978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:02.948652 30978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:02.949990 30978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:02.950084 30978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:02.950186 30978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:02.950258 30978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:02.979802 30978 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:35189
I20260812 07:10:02.979851 31214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:35189 every 8 connection(s)
I20260812 07:10:02.981011 30978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:02.984760 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30978
I20260812 07:10:02.984934 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:02.985566 22229 external_mini_cluster.cc:1428] Setting key c0c210e2ca46d06136d42ebb4d2f396c
I20260812 07:10:02.988271 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39941
--builtin_ntp_servers=127.21.181.84:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 07:10:02.988880 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:02.983265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:38847 (local address 127.21.181.126:39941)
0812 07:10:02.983472 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:02.983478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:02.984814 (+ 1336us) server_negotiation.cc:407] Connection header received
0812 07:10:02.986737 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:02.986744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:02.986804 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:02.986889 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:02.988131 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:02.988137 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:02.988141 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:02.988159 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:02.988192 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:02.988203 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:02.988310 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:02.988554 (+ 244us) server_negotiation.cc:299] Negotiation successful
0812 07:10:02.988623 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:10:02.989715 31218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:02.983773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39941 (local address 127.21.181.65:38847)
0812 07:10:02.984661 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:02.984704 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:02.986487 (+ 1783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:02.987123 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:02.987131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:02.987216 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:02.987832 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:02.987834 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:02.987868 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:02.987948 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:02.988292 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:02.988302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:02.988309 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:10:02.988616 (+ 307us) client_negotiation.cc:240] Negotiation successful
0812 07:10:02.988846 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":292,"threads_started":1}
I20260812 07:10:02.991040 31216 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39941
I20260812 07:10:02.991334 31216 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:02.991953 31216 heartbeater.cc:511] Master 127.21.181.126:39941 requested a full tablet report, sending...
I20260812 07:10:02.994108 30883 ts_manager.cc:194] Registered new tserver with Master: 101e86557f984b519580eb046c606edd (127.21.181.65:35189)
W20260812 07:10:03.303092 31221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:03.303336 31221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:03.303465 31221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:03.310513 31221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:03.310603 31221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:03.310750 31221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:03.319386 31221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:03.320524 31221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:03.322160 31221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:03.333995 31229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:03.334103 31231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:03.334131 31234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:03.335012 31221 server_base.cc:1034] running on GCE node
I20260812 07:10:03.335527 31221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:03.336364 31221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:03.337546 31221 hybrid_clock.cc:648] HybridClock initialized: now 1786518603337498 us; error 45 us; skew 500 ppm
I20260812 07:10:03.337823 31221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:03.339792 31221 webserver.cc:460] Webserver started at http://127.21.181.66:42975/ using document root <none> and password file <none>
I20260812 07:10:03.340451 31221 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:03.340540 31221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:03.340845 31221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:03.343403 31221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "8db2db93895044a6b85d8f1212d5914b"
format_stamp: "Formatted at 2026-08-12 07:10:03 on dist-test-slave-lgr5"
server_key: "877e4b707da4410dc6a32fbc465e0edb"
server_key_iv: "6adeb84d810bbc8a380c419b2746a21c"
server_key_version: "encryptionkey@0"
I20260812 07:10:03.344185 31221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "8db2db93895044a6b85d8f1212d5914b"
format_stamp: "Formatted at 2026-08-12 07:10:03 on dist-test-slave-lgr5"
server_key: "877e4b707da4410dc6a32fbc465e0edb"
server_key_iv: "6adeb84d810bbc8a380c419b2746a21c"
server_key_version: "encryptionkey@0"
I20260812 07:10:03.349013 31221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260812 07:10:03.352116 31246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:03.353279 31221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:10:03.353422 31221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "8db2db93895044a6b85d8f1212d5914b"
format_stamp: "Formatted at 2026-08-12 07:10:03 on dist-test-slave-lgr5"
server_key: "877e4b707da4410dc6a32fbc465e0edb"
server_key_iv: "6adeb84d810bbc8a380c419b2746a21c"
server_key_version: "encryptionkey@0"
I20260812 07:10:03.353606 31221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:03.358700 31221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:03.359391 31221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:03.359618 31221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:03.360526 31221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:03.361796 31221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:03.361884 31221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:03.361999 31221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:03.362067 31221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:03.391372 31221 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:43687
I20260812 07:10:03.391439 31466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:43687 every 8 connection(s)
I20260812 07:10:03.392588 31221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:03.398523 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31221
I20260812 07:10:03.398693 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:03.399230 22229 external_mini_cluster.cc:1428] Setting key ad54615a578e6b27ec8905966c7424f1
I20260812 07:10:03.400324 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.394878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:52447 (local address 127.21.181.126:39941)
0812 07:10:03.395091 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.395097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.396454 (+ 1357us) server_negotiation.cc:407] Connection header received
0812 07:10:03.398046 (+ 1592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.398051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.398104 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.398189 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.399595 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.399600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.399603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.399619 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.399652 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.399663 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.399766 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.399959 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.400026 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:10:03.401124 31470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.395410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39941 (local address 127.21.181.66:52447)
0812 07:10:03.396304 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.396349 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.397818 (+ 1469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.398444 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.398455 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.398581 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.399250 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.399254 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.399305 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.399414 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.399756 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.399767 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.399775 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.399993 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.400254 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":257,"threads_started":1}
I20260812 07:10:03.402038 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39941
--builtin_ntp_servers=127.21.181.84:36789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260812 07:10:03.402326 31467 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39941
I20260812 07:10:03.402626 31467 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:03.403265 31467 heartbeater.cc:511] Master 127.21.181.126:39941 requested a full tablet report, sending...
I20260812 07:10:03.404755 30883 ts_manager.cc:194] Registered new tserver with Master: 8db2db93895044a6b85d8f1212d5914b (127.21.181.66:43687)
W20260812 07:10:03.720649 31472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:03.720891 31472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:03.721019 31472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:03.727873 31472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:03.727962 31472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:03.728119 31472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:03.736609 31472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39941
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:03.737704 31472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:03.739326 31472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:03.751210 31487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:03.751298 31486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:03.751271 31490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:03.752272 31472 server_base.cc:1034] running on GCE node
I20260812 07:10:03.752776 31472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:03.753612 31472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:03.754797 31472 hybrid_clock.cc:648] HybridClock initialized: now 1786518603754751 us; error 35 us; skew 500 ppm
I20260812 07:10:03.755079 31472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:03.757100 31472 webserver.cc:460] Webserver started at http://127.21.181.67:36303/ using document root <none> and password file <none>
I20260812 07:10:03.757740 31472 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:03.757838 31472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:03.758143 31472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:03.760779 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "6de62f345be943aa8f692e0bcc021b5f"
format_stamp: "Formatted at 2026-08-12 07:10:03 on dist-test-slave-lgr5"
server_key: "dbe18af755dce8ff20ccd9192a0d9ef5"
server_key_iv: "818a1ff09f3d7acb29e2885e77d0c5e0"
server_key_version: "encryptionkey@0"
I20260812 07:10:03.761515 31472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "6de62f345be943aa8f692e0bcc021b5f"
format_stamp: "Formatted at 2026-08-12 07:10:03 on dist-test-slave-lgr5"
server_key: "dbe18af755dce8ff20ccd9192a0d9ef5"
server_key_iv: "818a1ff09f3d7acb29e2885e77d0c5e0"
server_key_version: "encryptionkey@0"
I20260812 07:10:03.766120 31472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:10:03.769207 31499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:03.770352 31472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:03.770500 31472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "6de62f345be943aa8f692e0bcc021b5f"
format_stamp: "Formatted at 2026-08-12 07:10:03 on dist-test-slave-lgr5"
server_key: "dbe18af755dce8ff20ccd9192a0d9ef5"
server_key_iv: "818a1ff09f3d7acb29e2885e77d0c5e0"
server_key_version: "encryptionkey@0"
I20260812 07:10:03.770685 31472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:03.775683 31472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:03.776459 31472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:03.776696 31472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:03.777508 31472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:03.778764 31472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:03.778857 31472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:03.778967 31472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:03.779043 31472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:03.807685 31472 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:42439
I20260812 07:10:03.807739 31716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:42439 every 8 connection(s)
I20260812 07:10:03.808858 31472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:03.812166 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31472
I20260812 07:10:03.812322 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:03.812855 22229 external_mini_cluster.cc:1428] Setting key f1cba0dd7ff6c2d50ae6f3330027b4df
I20260812 07:10:03.817044 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.811174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:58019 (local address 127.21.181.126:39941)
0812 07:10:03.811382 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.811387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.812832 (+ 1445us) server_negotiation.cc:407] Connection header received
0812 07:10:03.815068 (+ 2236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.815074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.815136 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.815242 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.816460 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.816465 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.816468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.816483 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.816517 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.816529 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.816615 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.816771 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.816831 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:10:03.817854 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.811626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39941 (local address 127.21.181.67:58019)
0812 07:10:03.812684 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.812728 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.814770 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.815403 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.815410 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.815504 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.816195 (+ 691us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.816198 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.816232 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.816314 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.816602 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.816607 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.816612 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.816824 (+ 212us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.817009 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":290,"threads_started":1}
I20260812 07:10:03.819007 31719 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39941
I20260812 07:10:03.819260 31719 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:03.819836 31719 heartbeater.cc:511] Master 127.21.181.126:39941 requested a full tablet report, sending...
I20260812 07:10:03.821080 30883 ts_manager.cc:194] Registered new tserver with Master: 6de62f345be943aa8f692e0bcc021b5f (127.21.181.67:42439)
I20260812 07:10:03.827834 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:10:03.834326 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.831990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44420 (local address 127.21.181.126:39941)
0812 07:10:03.832165 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.832170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.832639 (+ 469us) server_negotiation.cc:407] Connection header received
0812 07:10:03.832782 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.832788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.832842 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.832916 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.833763 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.833769 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.833774 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.833792 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.833820 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.833831 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.833929 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.834026 (+ 97us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.834083 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:10:03.837806 30883 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44420:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:03.844218 30883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:03.864945 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.855713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36620 (local address 127.21.181.65:35189)
0812 07:10:03.856332 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.856341 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.861093 (+ 4752us) server_negotiation.cc:407] Connection header received
0812 07:10:03.861615 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.861623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.861824 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.861950 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.864034 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.864042 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.864055 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.864114 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.864176 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.864199 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.864329 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.864544 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.864647 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":312,"threads_started":1}
I20260812 07:10:03.865056 31739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.855132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:42439 (local address 127.0.0.1:39080)
0812 07:10:03.860966 (+ 5834us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.861001 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.861125 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.862826 (+ 1701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.862835 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.862922 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.863815 (+ 893us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.863819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.863837 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.863886 (+ 49us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.864430 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.864435 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.864439 (+ 4us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.864588 (+ 149us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.864655 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5606,"thread_start_us":5499,"threads_started":1}
I20260812 07:10:03.865684 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.855339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35189 (local address 127.0.0.1:36620)
0812 07:10:03.860964 (+ 5625us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.861001 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.861181 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.862920 (+ 1739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.862926 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.862950 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.863768 (+ 818us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.863772 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.863799 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.863871 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.864309 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.864319 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.864327 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.865184 (+ 857us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.865238 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5470,"thread_start_us":5227,"threads_started":1}
I20260812 07:10:03.866281 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.855330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39080 (local address 127.21.181.67:42439)
0812 07:10:03.860595 (+ 5265us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.860603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.861256 (+ 653us) server_negotiation.cc:407] Connection header received
0812 07:10:03.861325 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.861332 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.861555 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.861705 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.864073 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.864102 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.864117 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.864137 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.864194 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.864214 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.864371 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.865454 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.865542 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5095,"thread_start_us":4934,"threads_started":1}
I20260812 07:10:03.867798 31740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.861506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:43687 (local address 127.0.0.1:36062)
0812 07:10:03.864647 (+ 3141us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.864661 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.864761 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.865505 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.865510 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.865535 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.866146 (+ 611us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.866150 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.866165 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.866207 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.867177 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.867182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.867187 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.867334 (+ 147us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.867383 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3030,"thread_start_us":2881,"threads_started":1}
I20260812 07:10:03.868578 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.861658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.21.181.66:43687)
0812 07:10:03.864236 (+ 2578us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.864244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.864769 (+ 525us) server_negotiation.cc:407] Connection header received
0812 07:10:03.864873 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.864880 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.865185 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.865332 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.866566 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.866572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.866580 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.866599 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.866633 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.866646 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.866801 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.868155 (+ 1354us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.868248 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2404,"thread_start_us":2050,"threads_started":1}
I20260812 07:10:03.869140 31100 tablet_service.cc:1467] Processing CreateTablet for tablet 2098d8ea485b4a2a9ff4408da17a4f4f (DEFAULT_TABLE table=test-table [id=4c44efb1e0ec4d1295452ccc0e917d7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:03.869706 31599 tablet_service.cc:1467] Processing CreateTablet for tablet 2098d8ea485b4a2a9ff4408da17a4f4f (DEFAULT_TABLE table=test-table [id=4c44efb1e0ec4d1295452ccc0e917d7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:03.870539 31100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2098d8ea485b4a2a9ff4408da17a4f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:03.870848 31599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2098d8ea485b4a2a9ff4408da17a4f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:03.872102 31342 tablet_service.cc:1467] Processing CreateTablet for tablet 2098d8ea485b4a2a9ff4408da17a4f4f (DEFAULT_TABLE table=test-table [id=4c44efb1e0ec4d1295452ccc0e917d7d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:03.873499 31342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2098d8ea485b4a2a9ff4408da17a4f4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:03.881721 31746 tablet_bootstrap.cc:492] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Bootstrap starting.
I20260812 07:10:03.884752 31747 tablet_bootstrap.cc:492] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Bootstrap starting.
I20260812 07:10:03.884788 31746 tablet_bootstrap.cc:654] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:03.886396 31746 log.cc:824] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:03.886931 31748 tablet_bootstrap.cc:492] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Bootstrap starting.
I20260812 07:10:03.888448 31747 tablet_bootstrap.cc:654] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:03.889587 31747 log.cc:824] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:03.889866 31748 tablet_bootstrap.cc:654] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:03.890236 31746 tablet_bootstrap.cc:492] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: No bootstrap required, opened a new log
I20260812 07:10:03.890484 31746 ts_tablet_manager.cc:1397] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:10:03.891036 31748 log.cc:824] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:03.892565 31747 tablet_bootstrap.cc:492] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: No bootstrap required, opened a new log
I20260812 07:10:03.892818 31747 ts_tablet_manager.cc:1397] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:10:03.894101 31748 tablet_bootstrap.cc:492] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: No bootstrap required, opened a new log
I20260812 07:10:03.894345 31748 ts_tablet_manager.cc:1397] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260812 07:10:03.897709 31746 raft_consensus.cc:348] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.898164 31746 raft_consensus.cc:374] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:03.898293 31746 raft_consensus.cc:729] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6de62f345be943aa8f692e0bcc021b5f, State: Initialized, Role: FOLLOWER
I20260812 07:10:03.898962 31746 consensus_queue.cc:260] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.899958 31719 heartbeater.cc:503] Master 127.21.181.126:39941 was elected leader, sending a full tablet report...
I20260812 07:10:03.900573 31746 ts_tablet_manager.cc:1428] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Time spent starting tablet: real 0.010s user 0.011s sys 0.001s
I20260812 07:10:03.902716 31747 raft_consensus.cc:348] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.903143 31747 raft_consensus.cc:374] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:03.903283 31747 raft_consensus.cc:729] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 101e86557f984b519580eb046c606edd, State: Initialized, Role: FOLLOWER
I20260812 07:10:03.903923 31747 consensus_queue.cc:260] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.904165 31748 raft_consensus.cc:348] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.904659 31748 raft_consensus.cc:374] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:03.904799 31748 raft_consensus.cc:729] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8db2db93895044a6b85d8f1212d5914b, State: Initialized, Role: FOLLOWER
I20260812 07:10:03.905153 31216 heartbeater.cc:503] Master 127.21.181.126:39941 was elected leader, sending a full tablet report...
I20260812 07:10:03.905663 31747 ts_tablet_manager.cc:1428] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260812 07:10:03.905548 31748 consensus_queue.cc:260] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.906520 31467 heartbeater.cc:503] Master 127.21.181.126:39941 was elected leader, sending a full tablet report...
I20260812 07:10:03.907114 31748 ts_tablet_manager.cc:1428] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260812 07:10:03.980860 31752 raft_consensus.cc:482] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:03.981098 31752 raft_consensus.cc:504] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.982616 31752 leader_election.cc:288] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 101e86557f984b519580eb046c606edd (127.21.181.65:35189), 8db2db93895044a6b85d8f1212d5914b (127.21.181.66:43687)
W20260812 07:10:03.982969 31217 tablet.cc:2369] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:03.986080 31722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.982971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35189 (local address 127.21.181.67:59611)
0812 07:10:03.983845 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.983863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.983955 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.984593 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.984597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.984621 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.985256 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.985260 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.985278 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.985325 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.985698 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.985703 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.985708 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.985800 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.985850 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":755}
I20260812 07:10:03.986158 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.983947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:59611 (local address 127.21.181.65:35189)
0812 07:10:03.984138 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.984144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.984168 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:10:03.984231 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.984236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.984299 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.984403 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.985482 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.985487 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.985492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.985511 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.985546 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.985560 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.985651 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.985829 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.985892 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:10:03.986447 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.983523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57359 (local address 127.21.181.66:43687)
0812 07:10:03.984182 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:03.984188 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:03.984637 (+ 449us) server_negotiation.cc:407] Connection header received
0812 07:10:03.984761 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:03.984766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:03.984823 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:03.984911 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:03.985739 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:03.985745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:03.985748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:03.985766 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:03.985802 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:03.985818 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:03.985933 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:03.986157 (+ 224us) server_negotiation.cc:299] Negotiation successful
0812 07:10:03.986219 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516}
I20260812 07:10:03.987109 31755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:03.983387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:43687 (local address 127.21.181.67:57359)
0812 07:10:03.984542 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:03.984559 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:03.984653 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:03.984944 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:03.984948 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:03.984972 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:03.985542 (+ 570us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:03.985545 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:03.985558 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:03.985595 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:03.985999 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:03.986004 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:03.986009 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:10:03.986690 (+ 681us) client_negotiation.cc:240] Negotiation successful
0812 07:10:03.986737 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1029,"thread_start_us":833,"threads_started":1}
I20260812 07:10:03.987452 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2098d8ea485b4a2a9ff4408da17a4f4f" candidate_uuid: "6de62f345be943aa8f692e0bcc021b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101e86557f984b519580eb046c606edd" is_pre_election: true
I20260812 07:10:03.988024 31138 raft_consensus.cc:2454] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de62f345be943aa8f692e0bcc021b5f in term 0.
I20260812 07:10:03.988029 31376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2098d8ea485b4a2a9ff4408da17a4f4f" candidate_uuid: "6de62f345be943aa8f692e0bcc021b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db2db93895044a6b85d8f1212d5914b" is_pre_election: true
I20260812 07:10:03.988482 31376 raft_consensus.cc:2454] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6de62f345be943aa8f692e0bcc021b5f in term 0.
I20260812 07:10:03.988961 31507 leader_election.cc:302] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 101e86557f984b519580eb046c606edd, 6de62f345be943aa8f692e0bcc021b5f; no voters:
I20260812 07:10:03.989351 31752 raft_consensus.cc:2781] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:03.989499 31752 raft_consensus.cc:482] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:03.989609 31752 raft_consensus.cc:3037] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:03.991737 31752 raft_consensus.cc:504] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:03.992337 31752 leader_election.cc:288] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [CANDIDATE]: Term 1 election: Requested vote from peers 101e86557f984b519580eb046c606edd (127.21.181.65:35189), 8db2db93895044a6b85d8f1212d5914b (127.21.181.66:43687)
I20260812 07:10:03.992702 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2098d8ea485b4a2a9ff4408da17a4f4f" candidate_uuid: "6de62f345be943aa8f692e0bcc021b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "101e86557f984b519580eb046c606edd"
I20260812 07:10:03.992849 31376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2098d8ea485b4a2a9ff4408da17a4f4f" candidate_uuid: "6de62f345be943aa8f692e0bcc021b5f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8db2db93895044a6b85d8f1212d5914b"
I20260812 07:10:03.992899 31138 raft_consensus.cc:3037] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:03.993057 31376 raft_consensus.cc:3037] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:03.996068 31376 raft_consensus.cc:2454] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de62f345be943aa8f692e0bcc021b5f in term 1.
I20260812 07:10:03.996068 31138 raft_consensus.cc:2454] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6de62f345be943aa8f692e0bcc021b5f in term 1.
I20260812 07:10:03.996430 31507 leader_election.cc:302] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6de62f345be943aa8f692e0bcc021b5f, 8db2db93895044a6b85d8f1212d5914b; no voters:
I20260812 07:10:03.996680 31752 raft_consensus.cc:2781] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:03.997097 31752 raft_consensus.cc:686] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 1 LEADER]: Becoming Leader. State: Replica: 6de62f345be943aa8f692e0bcc021b5f, State: Running, Role: LEADER
I20260812 07:10:03.997527 31752 consensus_queue.cc:237] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } }
I20260812 07:10:04.001993 30882 catalog_manager.cc:5495] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f reported cstate change: term changed from 0 to 1, leader changed from <none> to 6de62f345be943aa8f692e0bcc021b5f (127.21.181.67). New cstate: current_term: 1 leader_uuid: "6de62f345be943aa8f692e0bcc021b5f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6de62f345be943aa8f692e0bcc021b5f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 42439 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 } health_report { overall_health: UNKNOWN } } }
I20260812 07:10:04.026228 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:04.024659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.21.181.67:42439)
0812 07:10:04.024800 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:04.024804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:04.024820 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:10:04.024932 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:04.024935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:04.025016 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:04.025094 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:04.025814 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:04.025818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:04.025821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:04.025832 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:04.025855 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:04.025865 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:04.025948 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:04.026017 (+ 69us) server_negotiation.cc:299] Negotiation successful
0812 07:10:04.026058 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:10:04.033772 31376 raft_consensus.cc:1261] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 1 FOLLOWER]: Refusing update from remote peer 6de62f345be943aa8f692e0bcc021b5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:04.033772 31138 raft_consensus.cc:1261] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 1 FOLLOWER]: Refusing update from remote peer 6de62f345be943aa8f692e0bcc021b5f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:04.034598 31756 consensus_queue.cc:1035] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "8db2db93895044a6b85d8f1212d5914b" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 43687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:04.034904 31752 consensus_queue.cc:1035] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [LEADER]: Connected to new peer: Peer: permanent_uuid: "101e86557f984b519580eb046c606edd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:04.045256 31763 mvcc.cc:205] Tried to move back new op lower bound from 7317580202113220608 to 7317580201977262080. Current Snapshot: MvccSnapshot[applied={T|T < 7317580202113220608}]
I20260812 07:10:04.056881 30882 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:04.057114 30882 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44420:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:04.060545 30882 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33524a6a249b441490bbe2af75ae9d91: Sending DeleteTablet for 3 replicas of tablet 2098d8ea485b4a2a9ff4408da17a4f4f
I20260812 07:10:04.061873 31100 tablet_service.cc:1514] Processing DeleteTablet for tablet 2098d8ea485b4a2a9ff4408da17a4f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:04 UTC) from {username='slave'} at 127.0.0.1:36620
I20260812 07:10:04.061992 31342 tablet_service.cc:1514] Processing DeleteTablet for tablet 2098d8ea485b4a2a9ff4408da17a4f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:04 UTC) from {username='slave'} at 127.0.0.1:36062
I20260812 07:10:04.062285 31599 tablet_service.cc:1514] Processing DeleteTablet for tablet 2098d8ea485b4a2a9ff4408da17a4f4f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:04 UTC) from {username='slave'} at 127.0.0.1:39080
I20260812 07:10:04.064347 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30978
I20260812 07:10:04.077255 31787 tablet_replica.cc:330] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: stopping tablet replica
I20260812 07:10:04.077673 31787 raft_consensus.cc:2229] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:04.078159 31787 raft_consensus.cc:2258] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:04.082105 31791 tablet_replica.cc:330] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: stopping tablet replica
I20260812 07:10:04.082326 31787 ts_tablet_manager.cc:1905] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:04.082495 31791 raft_consensus.cc:2229] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:04.083073 31791 raft_consensus.cc:2258] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:04.084486 31738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:04.065817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36626 (local address 127.21.181.65:35189)
0812 07:10:04.065979 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:04.065985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:04.080531 (+ 14546us) server_negotiation.cc:407] Connection header received
0812 07:10:04.080933 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:04.080939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:04.081018 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:04.081091 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:04.083687 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:04.083693 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:04.083701 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:04.083719 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:04.083756 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:04.083771 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:04.083857 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:04.084130 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 07:10:04.084199 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:10:04.085409 31207 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:04.086216 31789 tablet_replica.cc:330] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: stopping tablet replica
I20260812 07:10:04.086609 31789 raft_consensus.cc:2229] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:04.087075 31789 raft_consensus.cc:2258] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:04.087116 31791 ts_tablet_manager.cc:1905] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:04.089131 31789 ts_tablet_manager.cc:1905] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:04.094378 31791 ts_tablet_manager.cc:1918] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:04.094614 31791 log.cc:1196] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/2098d8ea485b4a2a9ff4408da17a4f4f
I20260812 07:10:04.095249 31791 ts_tablet_manager.cc:1939] T 2098d8ea485b4a2a9ff4408da17a4f4f P 6de62f345be943aa8f692e0bcc021b5f: Deleting consensus metadata
I20260812 07:10:04.095949 31789 ts_tablet_manager.cc:1918] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:04.096138 31789 log.cc:1196] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/2098d8ea485b4a2a9ff4408da17a4f4f
I20260812 07:10:04.096662 31789 ts_tablet_manager.cc:1939] T 2098d8ea485b4a2a9ff4408da17a4f4f P 8db2db93895044a6b85d8f1212d5914b: Deleting consensus metadata
I20260812 07:10:04.097170 30854 catalog_manager.cc:4841] TS 6de62f345be943aa8f692e0bcc021b5f (127.21.181.67:42439): tablet 2098d8ea485b4a2a9ff4408da17a4f4f (table test-table [id=4c44efb1e0ec4d1295452ccc0e917d7d]) successfully deleted
I20260812 07:10:04.098850 30853 catalog_manager.cc:4841] TS 8db2db93895044a6b85d8f1212d5914b (127.21.181.66:43687): tablet 2098d8ea485b4a2a9ff4408da17a4f4f (table test-table [id=4c44efb1e0ec4d1295452ccc0e917d7d]) successfully deleted
I20260812 07:10:04.248025 31787 ts_tablet_manager.cc:1918] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:04.248279 31787 log.cc:1196] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/2098d8ea485b4a2a9ff4408da17a4f4f
I20260812 07:10:04.248742 31787 ts_tablet_manager.cc:1939] T 2098d8ea485b4a2a9ff4408da17a4f4f P 101e86557f984b519580eb046c606edd: Deleting consensus metadata
I20260812 07:10:04.250471 30853 catalog_manager.cc:4841] TS 101e86557f984b519580eb046c606edd (127.21.181.65:35189): tablet 2098d8ea485b4a2a9ff4408da17a4f4f (table test-table [id=4c44efb1e0ec4d1295452ccc0e917d7d]) successfully deleted
I20260812 07:10:04.250798 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30978
I20260812 07:10:04.275831 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31221
I20260812 07:10:04.278664 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:04.276700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36072 (local address 127.21.181.66:43687)
0812 07:10:04.276901 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:04.276906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:04.276929 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:10:04.277030 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:04.277035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:04.277150 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:04.277258 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:04.278042 (+ 784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:04.278048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:04.278053 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:04.278073 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:04.278113 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:04.278129 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:04.278263 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:04.278310 (+ 47us) server_negotiation.cc:299] Negotiation successful
0812 07:10:04.278369 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 07:10:04.279313 31459 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:04.364563 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31221
I20260812 07:10:04.389338 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31472
I20260812 07:10:04.392418 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:04.390212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39104 (local address 127.21.181.67:42439)
0812 07:10:04.390384 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:04.390388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:04.390445 (+ 57us) server_negotiation.cc:407] Connection header received
0812 07:10:04.390673 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:04.390677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:04.390729 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:04.390825 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:04.391737 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:04.391741 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:04.391748 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:04.391766 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:04.391796 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:04.391807 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:04.391921 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:04.392060 (+ 139us) server_negotiation.cc:299] Negotiation successful
0812 07:10:04.392171 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:10:04.393115 31711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:04.510010 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31472
I20260812 07:10:04.535338 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30815
I20260812 07:10:04.536491 30938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:04.639384 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30815
2026-08-12T07:10:04Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2630 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260812 07:10:04.685221 22229 mini_oidc.cc:152] Starting JWKS server
I20260812 07:10:04.687642 22229 webserver.cc:460] Webserver started at https://127.0.0.1:38503/ using document root <none> and password file <none>
I20260812 07:10:04.687940 22229 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:10:04.688544 22229 webserver.cc:460] Webserver started at http://127.0.0.1:33273/ using document root <none> and password file <none>
2026-08-12T07:10:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:04Z Disabled control of system clock
I20260812 07:10:04.717790 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:40511
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:41385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:40511
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38503/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/testchainca.cert with env {}
W20260812 07:10:05.025942 31812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:05.026253 31812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:05.026417 31812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:05.033571 31812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:05.033687 31812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:05.033762 31812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:05.033852 31812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:05.033922 31812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:10:05.034014 31812 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:10:05.034076 31812 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38503/jwks.json
I20260812 07:10:05.043694 31812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:41385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:40511
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:40511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38503/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:05.044899 31812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:05.046188 31812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:05.057557 31821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:05.057588 31825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:05.057510 31822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:05.058347 31812 server_base.cc:1034] running on GCE node
I20260812 07:10:05.059054 31812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:05.060289 31812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:05.061491 31812 hybrid_clock.cc:648] HybridClock initialized: now 1786518605061444 us; error 37 us; skew 500 ppm
I20260812 07:10:05.061800 31812 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:05.063882 31812 webserver.cc:460] Webserver started at http://127.21.181.126:34591/ using document root <none> and password file <none>
I20260812 07:10:05.064533 31812 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:05.064636 31812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:05.064927 31812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:05.067624 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "571cc88e66474fdf8371beba5acee388"
format_stamp: "Formatted at 2026-08-12 07:10:05 on dist-test-slave-lgr5"
server_key: "3d00b928c72fc71154a2d2f2d134533d"
server_key_iv: "ca992dbce3c44aa541614bc902f86840"
server_key_version: "encryptionkey@0"
I20260812 07:10:05.068418 31812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "571cc88e66474fdf8371beba5acee388"
format_stamp: "Formatted at 2026-08-12 07:10:05 on dist-test-slave-lgr5"
server_key: "3d00b928c72fc71154a2d2f2d134533d"
server_key_iv: "ca992dbce3c44aa541614bc902f86840"
server_key_version: "encryptionkey@0"
I20260812 07:10:05.073192 31812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:10:05.076421 31835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:05.077715 31812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:10:05.077872 31812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "571cc88e66474fdf8371beba5acee388"
format_stamp: "Formatted at 2026-08-12 07:10:05 on dist-test-slave-lgr5"
server_key: "3d00b928c72fc71154a2d2f2d134533d"
server_key_iv: "ca992dbce3c44aa541614bc902f86840"
server_key_version: "encryptionkey@0"
I20260812 07:10:05.078060 31812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:05.103139 31812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:05.103904 31812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:05.104225 31812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:05.119964 31812 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:40511
I20260812 07:10:05.119987 31927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:40511 every 8 connection(s)
I20260812 07:10:05.121258 31812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:05.122584 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31812
I20260812 07:10:05.122803 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:05.123605 22229 external_mini_cluster.cc:1428] Setting key 172a9302ed05ed3b7e88f8d8fb1e7917
I20260812 07:10:05.125775 31928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:05.136595 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:05.125445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54444 (local address 127.21.181.126:40511)
0812 07:10:05.126363 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:05.126374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:05.126415 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:10:05.127998 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:05.128018 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:05.128502 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:05.128806 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:05.129372 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.130698 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.132540 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.133140 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.133259 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:05.134441 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:05.134448 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:05.134460 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:05.134986 (+ 526us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:05.135068 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:05.135088 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:05.135184 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:05.135388 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 07:10:05.135686 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":332,"threads_started":1}
I20260812 07:10:05.137944 31928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388: Bootstrap starting.
I20260812 07:10:05.140236 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:10:05.141119 31928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:05.142033 31928 log.cc:824] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:05.144610 31928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388: No bootstrap required, opened a new log
I20260812 07:10:05.152128 31928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571cc88e66474fdf8371beba5acee388" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40511 } }
I20260812 07:10:05.152467 31928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:05.152559 31928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571cc88e66474fdf8371beba5acee388, State: Initialized, Role: FOLLOWER
I20260812 07:10:05.153076 31928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571cc88e66474fdf8371beba5acee388" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40511 } }
I20260812 07:10:05.153285 31928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:05.153412 31928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:05.153558 31928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:05.155937 31928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571cc88e66474fdf8371beba5acee388" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40511 } }
I20260812 07:10:05.156436 31928 leader_election.cc:302] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 571cc88e66474fdf8371beba5acee388; no voters:
I20260812 07:10:05.156950 31928 leader_election.cc:288] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:05.157090 31934 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:05.157717 31934 raft_consensus.cc:686] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [term 1 LEADER]: Becoming Leader. State: Replica: 571cc88e66474fdf8371beba5acee388, State: Running, Role: LEADER
I20260812 07:10:05.158212 31934 consensus_queue.cc:237] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571cc88e66474fdf8371beba5acee388" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40511 } }
I20260812 07:10:05.158818 31928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:05.160892 31938 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 571cc88e66474fdf8371beba5acee388. Latest consensus state: current_term: 1 leader_uuid: "571cc88e66474fdf8371beba5acee388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571cc88e66474fdf8371beba5acee388" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40511 } } }
I20260812 07:10:05.161016 31937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "571cc88e66474fdf8371beba5acee388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "571cc88e66474fdf8371beba5acee388" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40511 } } }
I20260812 07:10:05.161295 31938 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:05.161320 31937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:05.163313 31947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:05.167292 31947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:05.174649 31947 catalog_manager.cc:1347] Generated new cluster ID: e14e8ea7c0914accab862bbc996a2c0a
I20260812 07:10:05.174741 31947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:05.187239 31947 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:05.188541 31947 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:05.197777 31947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 571cc88e66474fdf8371beba5acee388: Generated new TSK 0
I20260812 07:10:05.198664 31947 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 07:10:05.234341 31977 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260812 07:10:05.234474 31977 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 07:10:05.235592 31931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:05.213922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54452 (local address 127.21.181.126:40511)
0812 07:10:05.214100 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:05.214105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:05.214900 (+ 795us) server_negotiation.cc:407] Connection header received
0812 07:10:05.215174 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:05.215179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:05.215264 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:05.215407 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:10:05.215879 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.217029 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.218442 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.219156 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.219279 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:05.219729 (+ 450us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:10:05.234938 (+ 15209us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 07:10:05.235288 (+ 350us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54452: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 07:10:05.235710 31931 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54452: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260812 07:10:05.236804 31974 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:05.214203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:40511 (local address 127.0.0.1:54452)
0812 07:10:05.214794 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:05.214809 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:05.214948 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:05.215414 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:05.215417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:05.215427 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:10:05.215744 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:05.215753 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.217171 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.217176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:05.218303 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:05.218316 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.219301 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.219306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:05.219415 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:05.219471 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:10:05.235183 (+ 15712us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0812 07:10:05.235415 (+ 232us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.181.126:40511: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":311,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.21.181.126:40511: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38503/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260812 07:10:05.238231 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31812
I20260812 07:10:05.239419 31922 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:05.328474 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31812
2026-08-12T07:10:05Z chronyd exiting
I20260812 07:10:05.382503 22229 test_util.cc:175] -----------------------------------------------
I20260812 07:10:05.382639 22229 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786518562800324-22229-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (701 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260812 07:10:05.386312 22229 mini_oidc.cc:152] Starting JWKS server
I20260812 07:10:05.388044 22229 webserver.cc:460] Webserver started at https://127.0.0.1:40135/ using document root <none> and password file <none>
I20260812 07:10:05.388278 22229 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:10:05.388864 22229 webserver.cc:460] Webserver started at http://127.0.0.1:45693/ using document root <none> and password file <none>
2026-08-12T07:10:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:05Z Disabled control of system clock
I20260812 07:10:05.419013 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34793
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:43545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:34793
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40135/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/testchainca.cert with env {}
W20260812 07:10:05.727813 31990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:05.728112 31990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:05.728248 31990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:05.735116 31990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:05.735219 31990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:05.735284 31990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:05.735363 31990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:05.735425 31990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:10:05.735510 31990 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:10:05.735562 31990 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40135/jwks.json
I20260812 07:10:05.745035 31990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:43545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:34793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40135/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:05.746201 31990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:05.747411 31990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:05.758437 31997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:05.758441 31998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:05.758371 32002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:05.759282 31990 server_base.cc:1034] running on GCE node
I20260812 07:10:05.759959 31990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:05.761211 31990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:05.762406 31990 hybrid_clock.cc:648] HybridClock initialized: now 1786518605762362 us; error 31 us; skew 500 ppm
I20260812 07:10:05.762703 31990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:05.768965 31990 webserver.cc:460] Webserver started at http://127.21.181.126:40503/ using document root <none> and password file <none>
I20260812 07:10:05.769594 31990 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:05.769696 31990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:05.770000 31990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:05.772725 31990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "f3ba739b8687494e929150f94c7cf3cd"
format_stamp: "Formatted at 2026-08-12 07:10:05 on dist-test-slave-lgr5"
server_key: "da260339a578bfc3fe8251649c3eb5f6"
server_key_iv: "93b97528d17a52d0afad9e73fd955970"
server_key_version: "encryptionkey@0"
I20260812 07:10:05.773483 31990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "f3ba739b8687494e929150f94c7cf3cd"
format_stamp: "Formatted at 2026-08-12 07:10:05 on dist-test-slave-lgr5"
server_key: "da260339a578bfc3fe8251649c3eb5f6"
server_key_iv: "93b97528d17a52d0afad9e73fd955970"
server_key_version: "encryptionkey@0"
I20260812 07:10:05.778223 31990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:10:05.781407 32011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:05.782593 31990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:10:05.782754 31990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "f3ba739b8687494e929150f94c7cf3cd"
format_stamp: "Formatted at 2026-08-12 07:10:05 on dist-test-slave-lgr5"
server_key: "da260339a578bfc3fe8251649c3eb5f6"
server_key_iv: "93b97528d17a52d0afad9e73fd955970"
server_key_version: "encryptionkey@0"
I20260812 07:10:05.782938 31990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:05.798851 31990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:05.799544 31990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:05.799803 31990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:05.816177 31990 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:34793
I20260812 07:10:05.816216 32102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:34793 every 8 connection(s)
I20260812 07:10:05.817349 31990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:05.820844 32105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:05.824478 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31990
I20260812 07:10:05.824644 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:05.825249 22229 external_mini_cluster.cc:1428] Setting key f00c29138f5295e9d4a87b4eb6149fdc
I20260812 07:10:05.831046 32105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd: Bootstrap starting.
I20260812 07:10:05.833905 32105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:05.834857 32105 log.cc:824] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:05.837574 32105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd: No bootstrap required, opened a new log
I20260812 07:10:05.837675 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:05.826947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50432 (local address 127.21.181.126:34793)
0812 07:10:05.827839 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:05.827849 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:05.827887 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:10:05.829828 (+ 1941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:05.829849 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:05.830255 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:05.830585 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:05.831089 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.832402 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.833619 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.834307 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.834470 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:05.835613 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:05.835620 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:05.835633 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:05.836173 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:05.836248 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:05.836272 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:05.836368 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:05.836572 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 07:10:05.836840 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":311,"threads_started":1}
I20260812 07:10:05.840982 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:10:05.844887 32105 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ba739b8687494e929150f94c7cf3cd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34793 } }
I20260812 07:10:05.845191 32105 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:05.845279 32105 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3ba739b8687494e929150f94c7cf3cd, State: Initialized, Role: FOLLOWER
I20260812 07:10:05.845803 32105 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ba739b8687494e929150f94c7cf3cd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34793 } }
I20260812 07:10:05.846014 32105 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:05.846140 32105 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:05.846287 32105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:05.848316 32105 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ba739b8687494e929150f94c7cf3cd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34793 } }
I20260812 07:10:05.848765 32105 leader_election.cc:302] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3ba739b8687494e929150f94c7cf3cd; no voters:
I20260812 07:10:05.849246 32105 leader_election.cc:288] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:05.849375 32113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:05.850528 32113 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [term 1 LEADER]: Becoming Leader. State: Replica: f3ba739b8687494e929150f94c7cf3cd, State: Running, Role: LEADER
I20260812 07:10:05.850975 32113 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ba739b8687494e929150f94c7cf3cd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34793 } }
I20260812 07:10:05.851346 32105 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:05.853407 32115 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3ba739b8687494e929150f94c7cf3cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ba739b8687494e929150f94c7cf3cd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34793 } } }
I20260812 07:10:05.853775 32115 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:05.854384 32116 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3ba739b8687494e929150f94c7cf3cd. Latest consensus state: current_term: 1 leader_uuid: "f3ba739b8687494e929150f94c7cf3cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3ba739b8687494e929150f94c7cf3cd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34793 } } }
I20260812 07:10:05.854789 32116 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:05.856554 32128 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:05.861286 32128 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:05.869021 32128 catalog_manager.cc:1347] Generated new cluster ID: 439282c4459241db8152e969cba24b14
I20260812 07:10:05.869120 32128 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:05.884155 32128 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:05.885365 32128 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:05.893020 32128 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3ba739b8687494e929150f94c7cf3cd: Generated new TSK 0
I20260812 07:10:05.893711 32128 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260812 07:10:05.923959 32152 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260812 07:10:05.924059 32152 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 07:10:05.925215 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:05.906026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.21.181.126:34793)
0812 07:10:05.906200 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:05.906207 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:05.906588 (+ 381us) server_negotiation.cc:407] Connection header received
0812 07:10:05.906785 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:05.906790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:05.906876 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:05.907036 (+ 160us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:10:05.907420 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.908539 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.909864 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.910551 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.910689 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:05.911086 (+ 397us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:10:05.924604 (+ 13518us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 07:10:05.924933 (+ 329us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":51}
W20260812 07:10:05.925240 32150 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:05.905875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34793 (local address 127.0.0.1:50446)
0812 07:10:05.906493 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:05.906513 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:05.906625 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:05.907028 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:05.907031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:05.907040 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:10:05.907270 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:05.907277 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.908657 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.908660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:05.909727 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:05.909738 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:05.910702 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:05.910706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:05.910826 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:05.910855 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:10:05.924791 (+ 13936us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0812 07:10:05.924960 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.181.126:34793: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":358,"threads_started":1}
W20260812 07:10:05.925345 32108 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50446: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40135/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260812 07:10:05.926173 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31990
I20260812 07:10:05.927237 32091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:06.020891 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31990
2026-08-12T07:10:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (695 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260812 07:10:06.081547 22229 mini_oidc.cc:152] Starting JWKS server
I20260812 07:10:06.083320 22229 webserver.cc:460] Webserver started at https://127.0.0.1:39743/ using document root <none> and password file <none>
I20260812 07:10:06.083586 22229 mini_oidc.cc:189] Starting OIDC Discovery server
I20260812 07:10:06.084229 22229 webserver.cc:460] Webserver started at http://127.0.0.1:38721/ using document root <none> and password file <none>
2026-08-12T07:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:06Z Disabled control of system clock
I20260812 07:10:06.112215 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43929
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:46757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:43929
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39743/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 07:10:06.425175 32162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:06.425451 32162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:06.425594 32162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:06.432943 32162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:06.433048 32162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:06.433120 32162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:06.433207 32162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:06.433285 32162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:10:06.433395 32162 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:10:06.433456 32162 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39743/jwks.json
I20260812 07:10:06.442235 32162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:43929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39743/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:06.443388 32162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:06.445120 32162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:06.456653 32172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:06.456686 32171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:06.456599 32178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:06.457401 32162 server_base.cc:1034] running on GCE node
I20260812 07:10:06.458019 32162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:06.459211 32162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:06.460399 32162 hybrid_clock.cc:648] HybridClock initialized: now 1786518606460349 us; error 40 us; skew 500 ppm
I20260812 07:10:06.460714 32162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:06.462757 32162 webserver.cc:460] Webserver started at http://127.21.181.126:39867/ using document root <none> and password file <none>
I20260812 07:10:06.463370 32162 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:06.463459 32162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:06.463762 32162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:06.466410 32162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "3b5ca6d999f64cd683419874c26dea15"
format_stamp: "Formatted at 2026-08-12 07:10:06 on dist-test-slave-lgr5"
server_key: "9d421d13506871796a2ad609be4844c4"
server_key_iv: "3b857778df83e3be404ee065b84dabb2"
server_key_version: "encryptionkey@0"
I20260812 07:10:06.467195 32162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "3b5ca6d999f64cd683419874c26dea15"
format_stamp: "Formatted at 2026-08-12 07:10:06 on dist-test-slave-lgr5"
server_key: "9d421d13506871796a2ad609be4844c4"
server_key_iv: "3b857778df83e3be404ee065b84dabb2"
server_key_version: "encryptionkey@0"
I20260812 07:10:06.471941 32162 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260812 07:10:06.475032 32185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:06.476330 32162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:06.476485 32162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "3b5ca6d999f64cd683419874c26dea15"
format_stamp: "Formatted at 2026-08-12 07:10:06 on dist-test-slave-lgr5"
server_key: "9d421d13506871796a2ad609be4844c4"
server_key_iv: "3b857778df83e3be404ee065b84dabb2"
server_key_version: "encryptionkey@0"
I20260812 07:10:06.476673 32162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:06.487083 32162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:06.487798 32162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:06.488148 32162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:06.504523 32162 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:43929
I20260812 07:10:06.504568 32276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:43929 every 8 connection(s)
I20260812 07:10:06.505725 32162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:06.506397 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 32162
I20260812 07:10:06.506605 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:06.507328 22229 external_mini_cluster.cc:1428] Setting key b76837397a425b534000fc2394626eee
I20260812 07:10:06.509500 32277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:06.521953 32280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:06.509176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34238 (local address 127.21.181.126:43929)
0812 07:10:06.510360 (+ 1184us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:06.510375 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:06.510486 (+ 111us) server_negotiation.cc:407] Connection header received
0812 07:10:06.512489 (+ 2003us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:06.512516 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:06.513065 (+ 549us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:06.513481 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:06.513968 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:06.515949 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:06.517210 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:06.518167 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:06.518352 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:06.519312 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:06.519323 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:06.519342 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:06.520019 (+ 677us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:06.520137 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:06.520168 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:06.520286 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:06.520479 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 07:10:06.520793 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":445,"threads_started":1}
I20260812 07:10:06.523207 32277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15: Bootstrap starting.
I20260812 07:10:06.525704 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:10:06.526971 32277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:06.528209 32277 log.cc:824] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:06.530812 32277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15: No bootstrap required, opened a new log
I20260812 07:10:06.537801 32277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5ca6d999f64cd683419874c26dea15" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43929 } }
I20260812 07:10:06.538110 32277 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:06.538179 32277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5ca6d999f64cd683419874c26dea15, State: Initialized, Role: FOLLOWER
I20260812 07:10:06.538632 32277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5ca6d999f64cd683419874c26dea15" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43929 } }
I20260812 07:10:06.538828 32277 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:06.538920 32277 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:06.539041 32277 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:06.541042 32277 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5ca6d999f64cd683419874c26dea15" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43929 } }
I20260812 07:10:06.541460 32277 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b5ca6d999f64cd683419874c26dea15; no voters:
I20260812 07:10:06.541952 32277 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:06.542083 32283 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:06.542682 32283 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [term 1 LEADER]: Becoming Leader. State: Replica: 3b5ca6d999f64cd683419874c26dea15, State: Running, Role: LEADER
I20260812 07:10:06.543313 32283 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5ca6d999f64cd683419874c26dea15" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43929 } }
I20260812 07:10:06.544018 32277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:06.546305 32286 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5ca6d999f64cd683419874c26dea15. Latest consensus state: current_term: 1 leader_uuid: "3b5ca6d999f64cd683419874c26dea15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5ca6d999f64cd683419874c26dea15" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43929 } } }
I20260812 07:10:06.546272 32285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b5ca6d999f64cd683419874c26dea15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5ca6d999f64cd683419874c26dea15" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43929 } } }
I20260812 07:10:06.546728 32286 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:06.546761 32285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:06.547914 32298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:06.552696 32298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:06.561326 32298 catalog_manager.cc:1347] Generated new cluster ID: 701c6b442d594200a6d6592d1f15b1df
I20260812 07:10:06.561420 32298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:06.572438 32298 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:06.573995 32298 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:06.589377 32298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b5ca6d999f64cd683419874c26dea15: Generated new TSK 0
I20260812 07:10:06.590234 32298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:10:06.615605 32280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:06.607697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.21.181.126:43929)
0812 07:10:06.607888 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:06.607895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:06.608319 (+ 424us) server_negotiation.cc:407] Connection header received
0812 07:10:06.608525 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:06.608531 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:06.608622 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:06.608778 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:10:06.609178 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:06.610264 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:06.611524 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:06.612239 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:06.612376 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:06.612810 (+ 434us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:10:06.615006 (+ 2196us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39743/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 07:10:06.615328 (+ 322us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34240: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39743/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":53}
W20260812 07:10:06.615731 32280 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34240: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39743/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260812 07:10:06.617281 32327 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:06.607608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43929 (local address 127.0.0.1:34240)
0812 07:10:06.608226 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:06.608241 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:06.608354 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:06.608767 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:06.608770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:06.608779 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:10:06.609042 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:06.609050 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:06.610367 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:06.610370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:06.611389 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:06.611400 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:06.612413 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:06.612417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:06.612544 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:06.612575 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:10:06.615212 (+ 2637us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39743/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0812 07:10:06.616872 (+ 1660us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.181.126:43929: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39743/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":1376,"thread_start_us":316,"threads_started":1}
I20260812 07:10:06.619042 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 32162
I20260812 07:10:06.620105 32268 generic_service.cc:196] Checking for leaks (request via RPC)
E20260812 07:10:06.629786 32328 webserver.cc:533] Webserver: sslize failed:
E20260812 07:10:06.629889 32328 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260812 07:10:06.710350 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 32162
2026-08-12T07:10:06Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (690 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-12T07:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:06Z Disabled control of system clock
I20260812 07:10:06.802266 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43245
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:33419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:43245
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/testchainca.cert with env {}
W20260812 07:10:07.113979 32340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:07.114257 32340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:07.114387 32340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:07.121498 32340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:07.121600 32340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:07.121663 32340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:07.121740 32340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:07.121805 32340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260812 07:10:07.121889 32340 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260812 07:10:07.121939 32340 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260812 07:10:07.131625 32340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:43245
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:07.132874 32340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:07.134151 32340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:07.145586 32352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:07.145613 32350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:07.145877 32349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:07.146311 32340 server_base.cc:1034] running on GCE node
I20260812 07:10:07.147037 32340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:07.148443 32340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:07.149649 32340 hybrid_clock.cc:648] HybridClock initialized: now 1786518607149600 us; error 43 us; skew 500 ppm
I20260812 07:10:07.149978 32340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:07.152123 32340 webserver.cc:460] Webserver started at http://127.21.181.126:44157/ using document root <none> and password file <none>
I20260812 07:10:07.152801 32340 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:07.152904 32340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:07.153187 32340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:07.155970 32340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "033c9eb42c9246cc891ca26499f31fe6"
format_stamp: "Formatted at 2026-08-12 07:10:07 on dist-test-slave-lgr5"
server_key: "cebdca59bc545619edb698e739b7d7ca"
server_key_iv: "ba1a3f85fc2b81100824eb5a8825c21c"
server_key_version: "encryptionkey@0"
I20260812 07:10:07.156793 32340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "033c9eb42c9246cc891ca26499f31fe6"
format_stamp: "Formatted at 2026-08-12 07:10:07 on dist-test-slave-lgr5"
server_key: "cebdca59bc545619edb698e739b7d7ca"
server_key_iv: "ba1a3f85fc2b81100824eb5a8825c21c"
server_key_version: "encryptionkey@0"
I20260812 07:10:07.161613 32340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:10:07.164840 32360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:07.166050 32340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:10:07.166213 32340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "033c9eb42c9246cc891ca26499f31fe6"
format_stamp: "Formatted at 2026-08-12 07:10:07 on dist-test-slave-lgr5"
server_key: "cebdca59bc545619edb698e739b7d7ca"
server_key_iv: "ba1a3f85fc2b81100824eb5a8825c21c"
server_key_version: "encryptionkey@0"
I20260812 07:10:07.166405 32340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:07.177282 32340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:07.177994 32340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:07.178241 32340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:07.194378 32340 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:43245
I20260812 07:10:07.194396 32466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:43245 every 8 connection(s)
I20260812 07:10:07.195574 32340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:07.199015 32467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:07.199380 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 32340
I20260812 07:10:07.199565 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:07.200397 22229 external_mini_cluster.cc:1428] Setting key e497e073967e7c33c79cb2cd139dfde0
I20260812 07:10:07.213361 32467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6: Bootstrap starting.
I20260812 07:10:07.215246 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:07.202146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51958 (local address 127.21.181.126:43245)
0812 07:10:07.203200 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:07.203222 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:07.203272 (+ 50us) server_negotiation.cc:407] Connection header received
0812 07:10:07.205841 (+ 2569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:07.205869 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:07.206426 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:07.206844 (+ 418us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:07.207343 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:07.209170 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:07.210393 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:07.211424 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:07.211584 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:07.212620 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:07.212631 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:07.212648 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:07.213287 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:07.213398 (+ 111us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:07.213426 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:07.213543 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:07.213724 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 07:10:07.214028 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":378,"threads_started":1}
I20260812 07:10:07.216830 32467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:07.217900 32467 log.cc:824] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:07.218716 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:10:07.220515 32467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6: No bootstrap required, opened a new log
I20260812 07:10:07.227680 32467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c9eb42c9246cc891ca26499f31fe6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43245 } }
I20260812 07:10:07.227979 32467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:07.228091 32467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 033c9eb42c9246cc891ca26499f31fe6, State: Initialized, Role: FOLLOWER
I20260812 07:10:07.228659 32467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c9eb42c9246cc891ca26499f31fe6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43245 } }
I20260812 07:10:07.228899 32467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:07.229038 32467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:07.229166 32467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:07.231076 32467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c9eb42c9246cc891ca26499f31fe6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43245 } }
I20260812 07:10:07.231508 32467 leader_election.cc:302] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 033c9eb42c9246cc891ca26499f31fe6; no voters:
I20260812 07:10:07.231992 32467 leader_election.cc:288] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:07.232137 32475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:07.233264 32475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [term 1 LEADER]: Becoming Leader. State: Replica: 033c9eb42c9246cc891ca26499f31fe6, State: Running, Role: LEADER
I20260812 07:10:07.233711 32475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c9eb42c9246cc891ca26499f31fe6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43245 } }
I20260812 07:10:07.234097 32467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:07.236274 32477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "033c9eb42c9246cc891ca26499f31fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c9eb42c9246cc891ca26499f31fe6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43245 } } }
I20260812 07:10:07.236323 32478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 033c9eb42c9246cc891ca26499f31fe6. Latest consensus state: current_term: 1 leader_uuid: "033c9eb42c9246cc891ca26499f31fe6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "033c9eb42c9246cc891ca26499f31fe6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43245 } } }
I20260812 07:10:07.236642 32477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:07.236663 32478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:07.238130 32495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:07.242784 32495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:07.250456 32495 catalog_manager.cc:1347] Generated new cluster ID: fdc21781a5604952addbb449c8647a06
I20260812 07:10:07.250566 32495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:07.289464 32495 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:07.291195 32495 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:07.301172 32495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 033c9eb42c9246cc891ca26499f31fe6: Generated new TSK 0
I20260812 07:10:07.302142 32495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:10:07.333897 32473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:07.319605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.21.181.126:43245)
0812 07:10:07.319798 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:07.319805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:07.320275 (+ 470us) server_negotiation.cc:407] Connection header received
0812 07:10:07.320501 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:07.320505 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:07.320580 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:07.320721 (+ 141us) server_negotiation.cc:226] Negotiated authn=JWT
0812 07:10:07.321195 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:07.321893 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:07.323880 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:07.324395 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:07.324484 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:07.325005 (+ 521us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0812 07:10:07.333396 (+ 8391us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 07:10:07.333678 (+ 282us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51968: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":55}
W20260812 07:10:07.334132 32473 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51968: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260812 07:10:07.334231 32537 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:07.319565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43245 (local address 127.0.0.1:51968)
0812 07:10:07.320179 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:07.320198 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:07.320347 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:07.320744 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:07.320749 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:07.320763 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0812 07:10:07.321062 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:07.321075 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:07.322035 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:07.322041 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:07.323750 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:07.323766 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:07.324555 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:07.324561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:07.324739 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:07.324784 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0812 07:10:07.333614 (+ 8830us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0812 07:10:07.333853 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.21.181.126:43245: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":292,"threads_started":1}
I20260812 07:10:07.335635 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 32340
I20260812 07:10:07.336575 32460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:07.427678 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 32340
2026-08-12T07:10:07Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (691 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (394 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (410 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:08 dist-test-slave-lgr5 krb5kdc[32565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:08 dist-test-slave-lgr5 krb5kdc[32565](info): set up 2 sockets
Aug 12 07:10:08 dist-test-slave-lgr5 krb5kdc[32565](info): commencing operation
krb5kdc: starting...
W20260812 07:10:09.463788 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.001s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:10:09 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:10:09.666658 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39427
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:39427
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:09.982533 32595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:09.982807 32595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:09.982942 32595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:09.989964 32595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:09.990077 32595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:09.990159 32595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:09.990232 32595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:09.990290 32595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:09.999714 32595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:39427
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:10.000972 32595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:10.002216 32595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:10.013422 32610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:10.013546 32604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:10.013514 32605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:10.014283 32595 server_base.cc:1034] running on GCE node
I20260812 07:10:10.015005 32595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:10.016287 32595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:10.017493 32595 hybrid_clock.cc:648] HybridClock initialized: now 1786518610017434 us; error 40 us; skew 500 ppm
I20260812 07:10:10.017799 32595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:10 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:10.020670 32595 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:10:10.022032 32595 webserver.cc:460] Webserver started at http://127.21.181.126:44365/ using document root <none> and password file <none>
I20260812 07:10:10.022663 32595 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:10.022778 32595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:10.023130 32595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:10.025974 32595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "beda671513dc4364beb5037d5c6fe8ad"
format_stamp: "Formatted at 2026-08-12 07:10:10 on dist-test-slave-lgr5"
server_key: "32f71fffa94f446c01c554d31de13e22"
server_key_iv: "3e021c380102089b2769d5cec27996e5"
server_key_version: "encryptionkey@0"
I20260812 07:10:10.026711 32595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "beda671513dc4364beb5037d5c6fe8ad"
format_stamp: "Formatted at 2026-08-12 07:10:10 on dist-test-slave-lgr5"
server_key: "32f71fffa94f446c01c554d31de13e22"
server_key_iv: "3e021c380102089b2769d5cec27996e5"
server_key_version: "encryptionkey@0"
I20260812 07:10:10.031523 32595 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260812 07:10:10.034721 32620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:10.035907 32595 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:10:10.036064 32595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "beda671513dc4364beb5037d5c6fe8ad"
format_stamp: "Formatted at 2026-08-12 07:10:10 on dist-test-slave-lgr5"
server_key: "32f71fffa94f446c01c554d31de13e22"
server_key_iv: "3e021c380102089b2769d5cec27996e5"
server_key_version: "encryptionkey@0"
I20260812 07:10:10.036286 32595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:10.049333 32595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:10.052716 32595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:10.052950 32595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:10.069430 32595 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:39427
I20260812 07:10:10.069469 32736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:39427 every 8 connection(s)
I20260812 07:10:10.070632 32595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:10.072934 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 32595
I20260812 07:10:10.073158 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:10.073951 22229 external_mini_cluster.cc:1428] Setting key 18dd35d583656e462bef7ef937cb1408
I20260812 07:10:10.074605 32737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:10.087937 32737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad: Bootstrap starting.
Aug 12 07:10:10 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:10.090883 32737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:10.091926 32737 log.cc:824] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:10.094471 32737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad: No bootstrap required, opened a new log
I20260812 07:10:10.096053 32740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:10.075822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55130 (local address 127.21.181.126:39427)
0812 07:10:10.076559 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:10.076570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:10.076606 (+ 36us) server_negotiation.cc:407] Connection header received
0812 07:10:10.078904 (+ 2298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:10.078925 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:10.079290 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:10.079598 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:10.081288 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:10.082522 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:10.084178 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:10.084847 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:10.084975 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:10.089356 (+ 4381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:10.089364 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:10.089376 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:10.089417 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:10.092393 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:10.093032 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:10.093038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:10.093044 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:10.093203 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:10.093561 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:10.093564 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:10.093566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:10.094046 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:10.094316 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:10.094756 (+ 440us) server_negotiation.cc:299] Negotiation successful
0812 07:10:10.095151 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":348,"threads_started":1}
I20260812 07:10:10.102166 32737 raft_consensus.cc:348] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beda671513dc4364beb5037d5c6fe8ad" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39427 } }
I20260812 07:10:10.102547 32737 raft_consensus.cc:374] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:10.102653 32737 raft_consensus.cc:729] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: beda671513dc4364beb5037d5c6fe8ad, State: Initialized, Role: FOLLOWER
I20260812 07:10:10.103168 32737 consensus_queue.cc:260] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beda671513dc4364beb5037d5c6fe8ad" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39427 } }
I20260812 07:10:10.103374 32737 raft_consensus.cc:388] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:10.103490 32737 raft_consensus.cc:482] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:10.103643 32737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:10.105690 32737 raft_consensus.cc:504] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beda671513dc4364beb5037d5c6fe8ad" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39427 } }
I20260812 07:10:10.106160 32737 leader_election.cc:302] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: beda671513dc4364beb5037d5c6fe8ad; no voters:
I20260812 07:10:10.106746 32737 leader_election.cc:288] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:10.106850 32743 raft_consensus.cc:2781] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:10.107416 32743 raft_consensus.cc:686] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [term 1 LEADER]: Becoming Leader. State: Replica: beda671513dc4364beb5037d5c6fe8ad, State: Running, Role: LEADER
I20260812 07:10:10.107978 32743 consensus_queue.cc:237] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beda671513dc4364beb5037d5c6fe8ad" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39427 } }
I20260812 07:10:10.108970 32737 sys_catalog.cc:564] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:10.110788 32746 sys_catalog.cc:455] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "beda671513dc4364beb5037d5c6fe8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beda671513dc4364beb5037d5c6fe8ad" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39427 } } }
I20260812 07:10:10.111155 32746 sys_catalog.cc:458] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:10.112298 32748 sys_catalog.cc:455] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader beda671513dc4364beb5037d5c6fe8ad. Latest consensus state: current_term: 1 leader_uuid: "beda671513dc4364beb5037d5c6fe8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "beda671513dc4364beb5037d5c6fe8ad" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39427 } } }
I20260812 07:10:10.112605 32748 sys_catalog.cc:458] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:10.114817 32758 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:10.121011 32758 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:10.129590 32758 catalog_manager.cc:1347] Generated new cluster ID: fcf55976f3ea4bd49621bdec8361dccc
I20260812 07:10:10.129683 32758 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:10.142323 32758 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:10.143461 32758 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:10.155076 32758 catalog_manager.cc:5868] T 00000000000000000000000000000000 P beda671513dc4364beb5037d5c6fe8ad: Generated new TSK 0
I20260812 07:10:10.155860 32758 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:10:10.205092 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39427
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:10.534174 322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:10.534461 322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:10.534570 322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:10.541904 322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:10.542011 322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:10.542135 322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:10.551510 322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:10.552712 322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:10.553972 322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:10.566944 331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:10.566895 335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:10.566855 333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:10.568071 322 server_base.cc:1034] running on GCE node
I20260812 07:10:10.568657 322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:10.569499 322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:10.570696 322 hybrid_clock.cc:648] HybridClock initialized: now 1786518610570648 us; error 44 us; skew 500 ppm
I20260812 07:10:10.570982 322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:10 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:10.573972 322 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:10:10.575335 322 webserver.cc:460] Webserver started at http://127.21.181.65:36137/ using document root <none> and password file <none>
I20260812 07:10:10.575968 322 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:10.576123 322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:10.576419 322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:10.579161 322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "45943328cb6144fb8a6e5d9e03099dd1"
format_stamp: "Formatted at 2026-08-12 07:10:10 on dist-test-slave-lgr5"
server_key: "b05b7fadea225807c510935823b2075d"
server_key_iv: "b0a94caa93c78262c737062f8b971fdf"
server_key_version: "encryptionkey@0"
I20260812 07:10:10.579916 322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "45943328cb6144fb8a6e5d9e03099dd1"
format_stamp: "Formatted at 2026-08-12 07:10:10 on dist-test-slave-lgr5"
server_key: "b05b7fadea225807c510935823b2075d"
server_key_iv: "b0a94caa93c78262c737062f8b971fdf"
server_key_version: "encryptionkey@0"
I20260812 07:10:10.584879 322 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260812 07:10:10.588115 349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:10.589309 322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:10:10.589463 322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "45943328cb6144fb8a6e5d9e03099dd1"
format_stamp: "Formatted at 2026-08-12 07:10:10 on dist-test-slave-lgr5"
server_key: "b05b7fadea225807c510935823b2075d"
server_key_iv: "b0a94caa93c78262c737062f8b971fdf"
server_key_version: "encryptionkey@0"
I20260812 07:10:10.589651 322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:10.601848 322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:10.605371 322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:10.605624 322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:10.606459 322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:10.607689 322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:10.607766 322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:10.607882 322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:10.607939 322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:10.638756 322 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:37501
I20260812 07:10:10.638778 553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:37501 every 8 connection(s)
I20260812 07:10:10.640229 322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:10.645823 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 322
I20260812 07:10:10.646019 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:10.646883 22229 external_mini_cluster.cc:1428] Setting key 9a715587c008722def3ab97209982d77
Aug 12 07:10:10 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518610, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:10.660892 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:10.642848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:35237 (local address 127.21.181.126:39427)
0812 07:10:10.643262 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:10.643266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:10.644233 (+ 967us) server_negotiation.cc:407] Connection header received
0812 07:10:10.645964 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:10.645970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:10.646060 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:10.646163 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:10.648137 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:10.648990 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:10.651205 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:10.651589 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:10.651674 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:10.655608 (+ 3934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:10.655613 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:10.655616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:10.655647 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:10.658017 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:10.658695 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:10.658698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:10.658701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:10.658818 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:10.659224 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:10.659227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:10.659229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:10.659491 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:10.659623 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:10.660428 (+ 805us) server_negotiation.cc:299] Negotiation successful
0812 07:10:10.660614 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":223,"threads_started":1}
I20260812 07:10:10.662078 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:10.643316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39427 (local address 127.21.181.65:35237)
0812 07:10:10.644049 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:10.644113 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:10.645709 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:10.646457 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:10.646469 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:10.647213 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:10.647944 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:10.647972 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:10.649138 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:10.649144 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:10.651066 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:10.651085 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:10.651807 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:10.651816 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:10.652130 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:10.652844 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:10.652874 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:10.655402 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:10.658155 (+ 2753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:10.658163 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:10.658177 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:10.658574 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:10.658919 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:10.658924 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:10.658928 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:10.659110 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:10.659615 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:10.659622 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:10.659980 (+ 358us) client_negotiation.cc:769] Sending connection context
0812 07:10:10.660287 (+ 307us) client_negotiation.cc:240] Negotiation successful
0812 07:10:10.660770 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":295,"threads_started":1}
I20260812 07:10:10.663599 554 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39427
I20260812 07:10:10.663923 554 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:10.664743 554 heartbeater.cc:511] Master 127.21.181.126:39427 requested a full tablet report, sending...
I20260812 07:10:10.666839 32654 ts_manager.cc:194] Registered new tserver with Master: 45943328cb6144fb8a6e5d9e03099dd1 (127.21.181.65:37501)
I20260812 07:10:10.668330 32654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:35237
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:10:10.696493 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39427
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:11.008428 564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:11.008689 564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:11.008790 564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:11.015750 564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:11.015849 564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:11.016000 564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:11.025288 564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:11.026430 564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:11.027734 564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:11.040228 570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:11.040277 572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:11.040306 574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:11.040971 564 server_base.cc:1034] running on GCE node
I20260812 07:10:11.041504 564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:11.042341 564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:11.043534 564 hybrid_clock.cc:648] HybridClock initialized: now 1786518611043486 us; error 42 us; skew 500 ppm
I20260812 07:10:11.043828 564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.050974 564 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:10:11.052383 564 webserver.cc:460] Webserver started at http://127.21.181.66:34899/ using document root <none> and password file <none>
I20260812 07:10:11.053009 564 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:11.053125 564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:11.053417 564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:11.056227 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "1852e5a5707c42bebc10a67fdbd5874e"
format_stamp: "Formatted at 2026-08-12 07:10:11 on dist-test-slave-lgr5"
server_key: "df39bb8f581ec457d8952d1e378fa892"
server_key_iv: "984181b00dc2cf002b018757eb033a6d"
server_key_version: "encryptionkey@0"
I20260812 07:10:11.056988 564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "1852e5a5707c42bebc10a67fdbd5874e"
format_stamp: "Formatted at 2026-08-12 07:10:11 on dist-test-slave-lgr5"
server_key: "df39bb8f581ec457d8952d1e378fa892"
server_key_iv: "984181b00dc2cf002b018757eb033a6d"
server_key_version: "encryptionkey@0"
I20260812 07:10:11.061657 564 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:10:11.064795 585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:11.065935 564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:11.066090 564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1852e5a5707c42bebc10a67fdbd5874e"
format_stamp: "Formatted at 2026-08-12 07:10:11 on dist-test-slave-lgr5"
server_key: "df39bb8f581ec457d8952d1e378fa892"
server_key_iv: "984181b00dc2cf002b018757eb033a6d"
server_key_version: "encryptionkey@0"
I20260812 07:10:11.066262 564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:11.090413 564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:11.094167 564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:11.094393 564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:11.095278 564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:11.096477 564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:11.096551 564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:11.096643 564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:11.096685 564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:11.126066 564 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:33487
I20260812 07:10:11.126109 777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:33487 every 8 connection(s)
I20260812 07:10:11.127290 564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:11.135321 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 564
I20260812 07:10:11.135511 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:11.135939 22229 external_mini_cluster.cc:1428] Setting key f51391a57234ee7df2bf07341da582b8
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:11.146754 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.129674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:51001 (local address 127.21.181.126:39427)
0812 07:10:11.129888 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.129894 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.131038 (+ 1144us) server_negotiation.cc:407] Connection header received
0812 07:10:11.132827 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.132831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.132903 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.132987 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.134354 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.135433 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.137135 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.137561 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.137687 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.141462 (+ 3775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.141467 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.141470 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.141512 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.143889 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.144602 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.144606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.144608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.144744 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.145159 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.145163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.145165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.145411 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.145557 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.146195 (+ 638us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.146415 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:10:11.147622 782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.130175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39427 (local address 127.21.181.66:51001)
0812 07:10:11.130882 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:11.130919 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:11.132585 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:11.133188 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:11.133195 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:11.133697 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:11.134226 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:11.134243 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.135586 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.135591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:11.136985 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:11.137008 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.137771 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.137778 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:11.138079 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.138763 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:11.138819 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:11.141258 (+ 2439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:11.144024 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:11.144033 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:11.144048 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:11.144482 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:11.144846 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:11.144851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:11.144855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:11.145047 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:11.145533 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:11.145542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:11.145841 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 07:10:11.146068 (+ 227us) client_negotiation.cc:240] Negotiation successful
0812 07:10:11.146541 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":283,"threads_started":1}
I20260812 07:10:11.148918 779 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39427
I20260812 07:10:11.149222 779 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:11.149999 779 heartbeater.cc:511] Master 127.21.181.126:39427 requested a full tablet report, sending...
I20260812 07:10:11.151548 32654 ts_manager.cc:194] Registered new tserver with Master: 1852e5a5707c42bebc10a67fdbd5874e (127.21.181.66:33487)
I20260812 07:10:11.152663 32654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:51001
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:10:11.186401 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39427
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:11.500727 791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:11.500993 791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:11.501093 791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:11.507962 791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:11.508055 791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:11.508203 791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:11.517570 791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39427
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:11.518713 791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:11.519958 791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:11.532272 800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:11.532320 805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:11.532353 799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:11.533115 791 server_base.cc:1034] running on GCE node
I20260812 07:10:11.533638 791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:11.534479 791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:11.535688 791 hybrid_clock.cc:648] HybridClock initialized: now 1786518611535646 us; error 32 us; skew 500 ppm
I20260812 07:10:11.535983 791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.538902 791 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:10:11.540313 791 webserver.cc:460] Webserver started at http://127.21.181.67:38429/ using document root <none> and password file <none>
I20260812 07:10:11.540926 791 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:11.541040 791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:11.541313 791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:11.544015 791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "038aca9ad64e44acb3f5f22a5b1ef143"
format_stamp: "Formatted at 2026-08-12 07:10:11 on dist-test-slave-lgr5"
server_key: "683aec791457a44322a779784d91428c"
server_key_iv: "ee428dc20328374a6c55513d3e4b6e78"
server_key_version: "encryptionkey@0"
I20260812 07:10:11.544785 791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "038aca9ad64e44acb3f5f22a5b1ef143"
format_stamp: "Formatted at 2026-08-12 07:10:11 on dist-test-slave-lgr5"
server_key: "683aec791457a44322a779784d91428c"
server_key_iv: "ee428dc20328374a6c55513d3e4b6e78"
server_key_version: "encryptionkey@0"
I20260812 07:10:11.549768 791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 07:10:11.552958 815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:11.554133 791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:10:11.554276 791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "038aca9ad64e44acb3f5f22a5b1ef143"
format_stamp: "Formatted at 2026-08-12 07:10:11 on dist-test-slave-lgr5"
server_key: "683aec791457a44322a779784d91428c"
server_key_iv: "ee428dc20328374a6c55513d3e4b6e78"
server_key_version: "encryptionkey@0"
I20260812 07:10:11.554451 791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:11.568572 791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:11.572523 791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:11.572772 791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:11.573609 791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:11.574812 791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:11.574898 791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:11.575013 791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:11.575088 791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:11.604676 791 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:35733
I20260812 07:10:11.604722 1034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:35733 every 8 connection(s)
I20260812 07:10:11.605855 791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:11.614030 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 791
I20260812 07:10:11.614183 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:11.614609 22229 external_mini_cluster.cc:1428] Setting key 4210c6533e7d8e69088d535267bb68a6
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:11.624433 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.608176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:38323 (local address 127.21.181.126:39427)
0812 07:10:11.608384 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.608389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.609483 (+ 1094us) server_negotiation.cc:407] Connection header received
0812 07:10:11.611257 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.611262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.611356 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.611443 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.612960 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.613762 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.615467 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.616071 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.616220 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.619577 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.619582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.619585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.619618 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.622038 (+ 2420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.622631 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.622634 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.622637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.622750 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.623072 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.623075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.623077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.623335 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.623446 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.623992 (+ 546us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.624197 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:11.625304 1038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.608626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39427 (local address 127.21.181.67:38323)
0812 07:10:11.609334 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:11.609376 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:11.610993 (+ 1617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:11.611689 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:11.611697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:11.612249 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:11.612785 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:11.612802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.613919 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.613924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:11.615349 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:11.615362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.616292 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.616299 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:11.616583 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.617232 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:11.617261 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:11.619396 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:11.622145 (+ 2749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:11.622151 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:11.622161 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:11.622544 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:11.622824 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:11.622827 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:11.622829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:11.622997 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:11.623424 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:11.623429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:11.623715 (+ 286us) client_negotiation.cc:769] Sending connection context
0812 07:10:11.623943 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 07:10:11.624337 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":289,"threads_started":1}
I20260812 07:10:11.626464 1035 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39427
I20260812 07:10:11.626730 1035 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:11.627411 1035 heartbeater.cc:511] Master 127.21.181.126:39427 requested a full tablet report, sending...
I20260812 07:10:11.628906 32653 ts_manager.cc:194] Registered new tserver with Master: 038aca9ad64e44acb3f5f22a5b1ef143 (127.21.181.67:35733)
I20260812 07:10:11.629851 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:10:11.629978 32653 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:38323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/test-admin.keytab.
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.659265 22229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260812 07:10:11.664026 1048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:11.661647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.21.181.65:37501)
0812 07:10:11.662224 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.662233 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.662264 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:10:11.662338 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.662347 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.662552 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.662693 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.663616 (+ 923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43770: BlockingRecv error: recv got EOF from 127.0.0.1:43770 (error 108)
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":262,"threads_started":1}
W20260812 07:10:11.664058 22229 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.665556 22229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.666787 22229 init.cc:295] Successfully reacquired a new kerberos TGT
I20260812 07:10:11.671614 554 heartbeater.cc:503] Master 127.21.181.126:39427 was elected leader, sending a full tablet report...
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:10:11.681780 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.668425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43780 (local address 127.21.181.65:37501)
0812 07:10:11.668610 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.668616 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.668714 (+ 98us) server_negotiation.cc:407] Connection header received
0812 07:10:11.668877 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.668881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.668937 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.669013 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.669990 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.671057 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.672568 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.673159 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.673276 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.676609 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.676619 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.676632 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.676672 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.679109 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.679737 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.679744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.679746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.679867 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.680281 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.680285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.680287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.680640 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.680919 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.681282 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.681535 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 07:10:11.686858 1048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:11.685339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43794 (local address 127.21.181.65:37501)
0812 07:10:11.685472 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.685475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.685762 (+ 287us) server_negotiation.cc:407] Connection header received
0812 07:10:11.685891 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.685894 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.685953 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.686023 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.686648 (+ 625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43794: BlockingRecv error: recv got EOF from 127.0.0.1:43794 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260812 07:10:11.687461 22229 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.688756 22229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.689950 22229 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:10:11.702569 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.691045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.21.181.65:37501)
0812 07:10:11.691184 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.691189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.691475 (+ 286us) server_negotiation.cc:407] Connection header received
0812 07:10:11.691596 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.691601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.691652 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.691725 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.692670 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.693358 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.694575 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.694971 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.695091 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.698213 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.698218 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.698221 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.698247 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.700362 (+ 2115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.700918 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.700922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.700924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.701040 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.701370 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.701375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.701378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.701628 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.701862 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.702155 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.702341 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260812 07:10:11.706159 1048 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:11.704686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.21.181.65:37501)
0812 07:10:11.704832 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.704837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.705103 (+ 266us) server_negotiation.cc:407] Connection header received
0812 07:10:11.705222 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.705227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.705287 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.705360 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.705994 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43800: BlockingRecv error: recv got EOF from 127.0.0.1:43800 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260812 07:10:11.706396 22229 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786518562800324-22229-0/krb5kdc/krb5cc' of type FILE
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.707831 22229 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:11.709048 22229 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:10:11.722390 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.710239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43808 (local address 127.21.181.65:37501)
0812 07:10:11.710427 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.710433 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.710660 (+ 227us) server_negotiation.cc:407] Connection header received
0812 07:10:11.710806 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.710810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.710876 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.710951 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.711891 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.712638 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.713850 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.714275 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.714386 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.717830 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.717835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.717838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.717869 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.720345 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.720886 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.720890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.720892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.721027 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.721288 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.721291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.721293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.721524 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.721725 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.721906 (+ 181us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.722075 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:10:11.723906 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 322
I20260812 07:10:11.735163 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.724624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43820 (local address 127.21.181.65:37501)
0812 07:10:11.724807 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.724812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.725108 (+ 296us) server_negotiation.cc:407] Connection header received
0812 07:10:11.725256 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.725260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.725311 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.725382 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.726251 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.727207 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.728419 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.728809 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.728914 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.730708 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.730713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.730717 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.730745 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.732826 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.733386 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.733391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.733394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.733539 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.733875 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.733880 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.733884 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.734130 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.734381 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.734674 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.734862 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:10:11.735671 549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:11.817565 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 322
I20260812 07:10:11.846033 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 564
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:10:11.860526 1074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.846876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52114 (local address 127.21.181.66:33487)
0812 07:10:11.847425 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.847432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.847459 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:10:11.847535 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.847541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.847738 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.847877 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.848918 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.849954 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.851070 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.851594 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.851685 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.855237 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.855250 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.855264 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.855303 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.858160 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.858751 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.858758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.858760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.858891 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.859186 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.859189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.859191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.859541 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.859767 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.859983 (+ 216us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.860227 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":277,"threads_started":1}
I20260812 07:10:11.861837 770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:11.941514 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 564
I20260812 07:10:11.960074 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 791
Aug 12 07:10:11 dist-test-slave-lgr5 krb5kdc[32565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:10:11.977015 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:11.961129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33642 (local address 127.21.181.67:35733)
0812 07:10:11.961735 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:11.961742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:11.961771 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:10:11.961830 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:11.961837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:11.962037 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:11.962163 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:11.963152 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.964196 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.965344 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:11.965876 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:11.965967 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:11.969642 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:11.969653 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:11.969666 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:11.969701 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:11.973897 (+ 4196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.974724 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.974733 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.974737 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.974894 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:11.975368 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:11.975373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:11.975377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:11.975807 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:11.976151 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:11.976429 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:10:11.976679 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":298,"threads_started":1}
I20260812 07:10:11.978232 1027 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:12.059139 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 791
I20260812 07:10:12.079542 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 32595
I20260812 07:10:12.080703 32729 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:12.183120 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 32595
2026-08-12T07:10:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3960 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:12 dist-test-slave-lgr5 krb5kdc[1084](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:12 dist-test-slave-lgr5 krb5kdc[1084](info): set up 2 sockets
Aug 12 07:10:12 dist-test-slave-lgr5 krb5kdc[1084](info): commencing operation
krb5kdc: starting...
W20260812 07:10:13.428645 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:10:13 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab.
Entry for principal oryx/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab.
I20260812 07:10:13.632905 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:41905
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab
--principal=oryx/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:41905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:13.943740 1116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:13.944020 1116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:13.944175 1116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:13.951076 1116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:13.951177 1116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:13.951254 1116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:13.951332 1116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:13.951391 1116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:13.960628 1116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:41905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab
--principal=oryx/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:41905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:13.961762 1116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:13.962945 1116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:13.974336 1130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:13.974774 1132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:13.974874 1128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:13.976312 1116 server_base.cc:1034] running on GCE node
I20260812 07:10:13.977197 1116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:13.978803 1116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:13.980028 1116 hybrid_clock.cc:648] HybridClock initialized: now 1786518613979965 us; error 46 us; skew 500 ppm
I20260812 07:10:13.980373 1116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:13 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:13.983197 1116 init.cc:377] Logged in from keytab as oryx/127.21.181.126@KRBTEST.COM (short username oryx)
I20260812 07:10:13.984622 1116 webserver.cc:460] Webserver started at http://127.21.181.126:36695/ using document root <none> and password file <none>
I20260812 07:10:13.985244 1116 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:13.985338 1116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:13.985575 1116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:13.988443 1116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "ad16eee3ad2d49539a1039a65084d1ac"
format_stamp: "Formatted at 2026-08-12 07:10:13 on dist-test-slave-lgr5"
server_key: "2217643941a77a3457606b3cf6199b61"
server_key_iv: "7a2beac3465109dcf5086afb9f14c1b7"
server_key_version: "encryptionkey@0"
I20260812 07:10:13.989176 1116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "ad16eee3ad2d49539a1039a65084d1ac"
format_stamp: "Formatted at 2026-08-12 07:10:13 on dist-test-slave-lgr5"
server_key: "2217643941a77a3457606b3cf6199b61"
server_key_iv: "7a2beac3465109dcf5086afb9f14c1b7"
server_key_version: "encryptionkey@0"
I20260812 07:10:13.993850 1116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:10:13.997130 1150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:13.998445 1116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:13.998590 1116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "ad16eee3ad2d49539a1039a65084d1ac"
format_stamp: "Formatted at 2026-08-12 07:10:13 on dist-test-slave-lgr5"
server_key: "2217643941a77a3457606b3cf6199b61"
server_key_iv: "7a2beac3465109dcf5086afb9f14c1b7"
server_key_version: "encryptionkey@0"
I20260812 07:10:13.998764 1116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:14.014889 1116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:14.018435 1116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:14.018652 1116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:14.035676 1116 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:41905
I20260812 07:10:14.035705 1250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:41905 every 8 connection(s)
I20260812 07:10:14.037057 1116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:14.039860 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 1116
I20260812 07:10:14.040033 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:14.040700 22229 external_mini_cluster.cc:1428] Setting key 083d4e136b8d501e7d4a4116dc33b14b
I20260812 07:10:14.040710 1254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:14.053726 1254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Bootstrap starting.
Aug 12 07:10:14 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:14.057121 1254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:14.058159 1254 log.cc:824] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:14.061161 1254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: No bootstrap required, opened a new log
I20260812 07:10:14.064595 1260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:14.042601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42434 (local address 127.21.181.126:41905)
0812 07:10:14.043546 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:14.043565 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:14.044175 (+ 610us) server_negotiation.cc:407] Connection header received
0812 07:10:14.046475 (+ 2300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:14.046503 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:14.046853 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:14.047158 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:14.048913 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:14.050165 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:14.051885 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:14.052564 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:14.052681 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:14.057041 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:14.057048 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:14.057061 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:14.057098 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:14.060363 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:14.060991 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:14.060999 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:14.061006 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:14.061228 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:14.061591 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:14.061597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:14.061600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:14.062270 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:14.062595 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:14.062868 (+ 273us) server_negotiation.cc:299] Negotiation successful
0812 07:10:14.063295 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":394,"threads_started":1}
I20260812 07:10:14.069263 1254 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } }
I20260812 07:10:14.069573 1254 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:14.069664 1254 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad16eee3ad2d49539a1039a65084d1ac, State: Initialized, Role: FOLLOWER
I20260812 07:10:14.070178 1254 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } }
I20260812 07:10:14.070389 1254 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:14.070519 1254 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:14.070659 1254 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:14.072773 1254 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } }
I20260812 07:10:14.073217 1254 leader_election.cc:302] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad16eee3ad2d49539a1039a65084d1ac; no voters:
I20260812 07:10:14.073729 1254 leader_election.cc:288] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:14.073835 1262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:14.074431 1262 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 LEADER]: Becoming Leader. State: Replica: ad16eee3ad2d49539a1039a65084d1ac, State: Running, Role: LEADER
I20260812 07:10:14.074916 1262 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } }
I20260812 07:10:14.075562 1254 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:14.077766 1263 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } }
I20260812 07:10:14.078030 1263 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:14.079541 1271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:14.079414 1264 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad16eee3ad2d49539a1039a65084d1ac. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } }
I20260812 07:10:14.079725 1264 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:14.085201 1271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:14.093526 1271 catalog_manager.cc:1347] Generated new cluster ID: 82424b418be74224ace6b4218c4e4b12
I20260812 07:10:14.093626 1271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:14.115599 1271 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:14.117245 1271 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:14.123411 1271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Generated new TSK 0
I20260812 07:10:14.124185 1271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab.
Entry for principal oryx/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab.
I20260812 07:10:14.185497 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:41905
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:14.562731 1301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:14.562981 1301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:14.563097 1301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:14.570185 1301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:14.570295 1301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:14.570443 1301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:14.579842 1301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:14.581009 1301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:14.582300 1301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:14.594493 1310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:14.594558 1313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:14.594529 1309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:14.595490 1301 server_base.cc:1034] running on GCE node
I20260812 07:10:14.596036 1301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:14.596930 1301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:14.598133 1301 hybrid_clock.cc:648] HybridClock initialized: now 1786518614598078 us; error 36 us; skew 500 ppm
I20260812 07:10:14.598431 1301 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:14 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518614, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:14.601441 1301 init.cc:377] Logged in from keytab as oryx/127.21.181.65@KRBTEST.COM (short username oryx)
I20260812 07:10:14.602870 1301 webserver.cc:460] Webserver started at http://127.21.181.65:44515/ using document root <none> and password file <none>
I20260812 07:10:14.603523 1301 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:14.603636 1301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:14.603967 1301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:14.606864 1301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "c883c5578a264138870f8efc4d129bd2"
format_stamp: "Formatted at 2026-08-12 07:10:14 on dist-test-slave-lgr5"
server_key: "2dc708a1daa44afe30c6a20a6ee15ff3"
server_key_iv: "625c98449a29183a5b0e3af750d57077"
server_key_version: "encryptionkey@0"
I20260812 07:10:14.607612 1301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "c883c5578a264138870f8efc4d129bd2"
format_stamp: "Formatted at 2026-08-12 07:10:14 on dist-test-slave-lgr5"
server_key: "2dc708a1daa44afe30c6a20a6ee15ff3"
server_key_iv: "625c98449a29183a5b0e3af750d57077"
server_key_version: "encryptionkey@0"
I20260812 07:10:14.612659 1301 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 07:10:14.616017 1327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:14.617342 1301 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:10:14.617487 1301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "c883c5578a264138870f8efc4d129bd2"
format_stamp: "Formatted at 2026-08-12 07:10:14 on dist-test-slave-lgr5"
server_key: "2dc708a1daa44afe30c6a20a6ee15ff3"
server_key_iv: "625c98449a29183a5b0e3af750d57077"
server_key_version: "encryptionkey@0"
I20260812 07:10:14.617697 1301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:14.649107 1301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:14.652859 1301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:14.653126 1301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:14.654050 1301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:14.655371 1301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:14.655443 1301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:14.655575 1301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:14.655627 1301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:14.685341 1301 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:44169
I20260812 07:10:14.685388 1533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:44169 every 8 connection(s)
I20260812 07:10:14.686573 1301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:14.687942 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 1301
I20260812 07:10:14.688164 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:14.688998 22229 external_mini_cluster.cc:1428] Setting key 07ed228bf08e60d41aec882044cb75d9
Aug 12 07:10:14 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518614, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:14.707221 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:14.689049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:32773 (local address 127.21.181.126:41905)
0812 07:10:14.689925 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:14.689931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:14.690577 (+ 646us) server_negotiation.cc:407] Connection header received
0812 07:10:14.692911 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:14.692915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:14.692976 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:14.693072 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:14.694894 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:14.695604 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:14.697664 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:14.698051 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:14.698143 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:14.701797 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:14.701802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:14.701805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:14.701856 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:14.704347 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:14.705143 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:14.705146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:14.705148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:14.705261 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:14.705673 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:14.705676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:14.705678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:14.705915 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:14.706021 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:14.706815 (+ 794us) server_negotiation.cc:299] Negotiation successful
0812 07:10:14.706971 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":616,"threads_started":1}
I20260812 07:10:14.708658 1538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:14.689702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:32773)
0812 07:10:14.690437 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:14.690476 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:14.692629 (+ 2153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:14.693332 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:14.693343 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:14.694014 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:14.694746 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:14.694771 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:14.695773 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:14.695778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:14.697540 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:14.697555 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:14.698266 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:14.698273 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:14.698548 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:14.699189 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:14.699226 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:14.701560 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:14.704480 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:14.704489 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:14.704503 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:14.705044 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:14.705353 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:14.705357 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:14.705360 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:14.705583 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:14.706048 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:14.706057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:14.706468 (+ 411us) client_negotiation.cc:769] Sending connection context
0812 07:10:14.706745 (+ 277us) client_negotiation.cc:240] Negotiation successful
0812 07:10:14.707227 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":322,"threads_started":1}
I20260812 07:10:14.710096 1534 heartbeater.cc:347] Connected to a master server at 127.21.181.126:41905
I20260812 07:10:14.710366 1534 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:14.711067 1534 heartbeater.cc:511] Master 127.21.181.126:41905 requested a full tablet report, sending...
I20260812 07:10:14.713104 1184 ts_manager.cc:194] Registered new tserver with Master: c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:14.714509 1184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:32773
WARNING: no policy specified for oryx/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab.
Entry for principal oryx/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab.
I20260812 07:10:14.742219 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:41905
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:15.087251 1544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:15.087644 1544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:15.087807 1544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:15.098609 1544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:15.098747 1544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:15.098935 1544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:15.109000 1544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:15.110180 1544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:15.111435 1544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:15.123698 1551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:15.123674 1559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:15.124020 1550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:15.124629 1544 server_base.cc:1034] running on GCE node
I20260812 07:10:15.125195 1544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:15.126075 1544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:15.127269 1544 hybrid_clock.cc:648] HybridClock initialized: now 1786518615127222 us; error 41 us; skew 500 ppm
I20260812 07:10:15.127568 1544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:15 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518615, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:15.130517 1544 init.cc:377] Logged in from keytab as oryx/127.21.181.66@KRBTEST.COM (short username oryx)
I20260812 07:10:15.131816 1544 webserver.cc:460] Webserver started at http://127.21.181.66:44703/ using document root <none> and password file <none>
I20260812 07:10:15.132483 1544 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:15.132592 1544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:15.132836 1544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:15.135627 1544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "1050870805f441d287bd01e27acedd9a"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "72daff45d1e69427bcc0788abd157512"
server_key_iv: "2b5a877ee2fdcb2ce5e35912b423e1be"
server_key_version: "encryptionkey@0"
I20260812 07:10:15.136384 1544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "1050870805f441d287bd01e27acedd9a"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "72daff45d1e69427bcc0788abd157512"
server_key_iv: "2b5a877ee2fdcb2ce5e35912b423e1be"
server_key_version: "encryptionkey@0"
I20260812 07:10:15.141209 1544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:10:15.144330 1577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:15.145524 1544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:10:15.145654 1544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1050870805f441d287bd01e27acedd9a"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "72daff45d1e69427bcc0788abd157512"
server_key_iv: "2b5a877ee2fdcb2ce5e35912b423e1be"
server_key_version: "encryptionkey@0"
I20260812 07:10:15.145814 1544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:15.158345 1544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:15.161867 1544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:15.162103 1544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:15.162926 1544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:15.164191 1544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:15.164263 1544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:15.164352 1544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:15.164398 1544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:15.194314 1544 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:39441
I20260812 07:10:15.194352 1808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:39441 every 8 connection(s)
I20260812 07:10:15.195600 1544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:15.196882 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 1544
I20260812 07:10:15.197043 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:15.197582 22229 external_mini_cluster.cc:1428] Setting key 58f0d56ffbccbe0d96ea52a0973f5f38
Aug 12 07:10:15 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518615, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:15.216789 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:15.198112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:53073 (local address 127.21.181.126:41905)
0812 07:10:15.198322 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:15.198328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:15.199486 (+ 1158us) server_negotiation.cc:407] Connection header received
0812 07:10:15.201891 (+ 2405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:15.201895 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:15.201958 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:15.202043 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:15.203834 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.204577 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.206544 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.206953 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.207041 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:15.210781 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:15.210786 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:15.210790 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:15.210822 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:15.213782 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:15.214472 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:15.214478 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:15.214481 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:15.214627 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:15.215027 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:15.215032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:15.215035 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:15.215405 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:15.215538 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:15.216211 (+ 673us) server_negotiation.cc:299] Negotiation successful
0812 07:10:15.216465 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:10:15.217666 1811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:15.198567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:53073)
0812 07:10:15.199311 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:15.199397 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:15.201616 (+ 2219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:15.202293 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:15.202304 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:15.202961 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:15.203666 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:15.203688 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.204724 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.204729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:15.206427 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:15.206441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.207162 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.207169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:15.207449 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:15.208119 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:15.208146 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:15.210562 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:15.213928 (+ 3366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:15.213935 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:15.213946 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:15.214358 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:15.214718 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:15.214721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:15.214723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:15.214912 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:15.215514 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:15.215519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:15.215832 (+ 313us) client_negotiation.cc:769] Sending connection context
0812 07:10:15.216064 (+ 232us) client_negotiation.cc:240] Negotiation successful
0812 07:10:15.216492 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":300,"threads_started":1}
I20260812 07:10:15.219053 1809 heartbeater.cc:347] Connected to a master server at 127.21.181.126:41905
I20260812 07:10:15.219353 1809 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:15.220116 1809 heartbeater.cc:511] Master 127.21.181.126:41905 requested a full tablet report, sending...
I20260812 07:10:15.221701 1184 ts_manager.cc:194] Registered new tserver with Master: 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441)
I20260812 07:10:15.222875 1184 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:53073
WARNING: no policy specified for oryx/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab.
Entry for principal oryx/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab.
I20260812 07:10:15.252614 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:41905
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:15.571702 1815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:15.571964 1815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:15.572070 1815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:15.579107 1815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:15.579213 1815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:15.579342 1815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:15.588662 1815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:41905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:15.589810 1815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:15.591091 1815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:15.604113 1826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:15.604274 1836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:15.604389 1825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:15.605103 1815 server_base.cc:1034] running on GCE node
I20260812 07:10:15.605712 1815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:15.606647 1815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:15.607854 1815 hybrid_clock.cc:648] HybridClock initialized: now 1786518615607802 us; error 39 us; skew 500 ppm
I20260812 07:10:15.608211 1815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:15 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518615, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:15.611270 1815 init.cc:377] Logged in from keytab as oryx/127.21.181.67@KRBTEST.COM (short username oryx)
I20260812 07:10:15.612821 1815 webserver.cc:460] Webserver started at http://127.21.181.67:36473/ using document root <none> and password file <none>
I20260812 07:10:15.613493 1815 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:15.613622 1815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:15.613935 1815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:15.616936 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "c1d239dc082f41a4ae5eafce924400f3"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "d9b04901691d1c58c4af0b6163647ede"
server_key_iv: "700d07f24e0dfc77d889953945fcb31b"
server_key_version: "encryptionkey@0"
I20260812 07:10:15.617758 1815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "c1d239dc082f41a4ae5eafce924400f3"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "d9b04901691d1c58c4af0b6163647ede"
server_key_iv: "700d07f24e0dfc77d889953945fcb31b"
server_key_version: "encryptionkey@0"
I20260812 07:10:15.623342 1815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260812 07:10:15.626873 1847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:15.628155 1815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:15.628352 1815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "c1d239dc082f41a4ae5eafce924400f3"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "d9b04901691d1c58c4af0b6163647ede"
server_key_iv: "700d07f24e0dfc77d889953945fcb31b"
server_key_version: "encryptionkey@0"
I20260812 07:10:15.628609 1815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:15.670130 1815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:15.674192 1815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:15.674499 1815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:15.675524 1815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:15.676993 1815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:15.677076 1815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:15.677210 1815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:15.677269 1815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:15.709475 1815 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38483
I20260812 07:10:15.709511 2066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38483 every 8 connection(s)
I20260812 07:10:15.710742 1815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:15.714466 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 1815
I20260812 07:10:15.714644 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:15.715271 22229 external_mini_cluster.cc:1428] Setting key f39a632b43373672ee85214b494e54f4
I20260812 07:10:15.717404 1534 heartbeater.cc:503] Master 127.21.181.126:41905 was elected leader, sending a full tablet report...
Aug 12 07:10:15 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518615, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:15.730430 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:15.713150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57149 (local address 127.21.181.126:41905)
0812 07:10:15.713324 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:15.713329 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:15.714582 (+ 1253us) server_negotiation.cc:407] Connection header received
0812 07:10:15.716757 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:15.716762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:15.716870 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:15.716960 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:15.718834 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.719761 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.721611 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.722027 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.722131 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:15.725371 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:15.725376 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:15.725379 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:15.725412 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:15.727653 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:15.728326 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:15.728330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:15.728332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:15.728448 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:15.728779 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:15.728782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:15.728784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:15.729031 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:15.729136 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:15.729940 (+ 804us) server_negotiation.cc:299] Negotiation successful
0812 07:10:15.730131 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:10:15.731482 2070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:15.713612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:57149)
0812 07:10:15.714391 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:15.714439 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:15.716505 (+ 2066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:15.717213 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:15.717223 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:15.717856 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:15.718609 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:15.718630 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.719968 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.719972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:15.721466 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:15.721480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:15.722237 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:15.722245 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:15.722493 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:15.723120 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:15.723143 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:15.725193 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:15.727777 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:15.727784 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:15.727795 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:15.728234 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:15.728527 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:15.728530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:15.728532 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:15.728701 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:15.729158 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:15.729166 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:15.729557 (+ 391us) client_negotiation.cc:769] Sending connection context
0812 07:10:15.729826 (+ 269us) client_negotiation.cc:240] Negotiation successful
0812 07:10:15.730288 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":288,"threads_started":1}
I20260812 07:10:15.732718 2068 heartbeater.cc:347] Connected to a master server at 127.21.181.126:41905
I20260812 07:10:15.733047 2068 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:15.733759 2068 heartbeater.cc:511] Master 127.21.181.126:41905 requested a full tablet report, sending...
I20260812 07:10:15.735015 1183 ts_manager.cc:194] Registered new tserver with Master: c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483)
I20260812 07:10:15.735884 1183 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:57149
I20260812 07:10:15.742761 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.181.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.125.keytab.
Entry for principal oryx/127.21.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.125.keytab.
I20260812 07:10:15.793212 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:37625
--webserver_interface=127.21.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.125.keytab
--principal=oryx/127.21.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:41905,127.21.181.125:37625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:16.108701 2075 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:16.108954 2075 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:16.109078 2075 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:16.115939 2075 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:16.116031 2075 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:16.116117 2075 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:16.116190 2075 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:16.116254 2075 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:16.125619 2075 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:41905,127.21.181.125:37625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.125.keytab
--principal=oryx/127.21.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:37625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
W20260812 07:10:16.125931 2075 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.21.181.126:41905,127.21.181.125:37625'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260812 07:10:16.126884 2075 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:16.128149 2075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:16.139045 2082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:16.139169 2092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:16.139138 2085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:16.140198 2075 server_base.cc:1034] running on GCE node
I20260812 07:10:16.141026 2075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:16.142280 2075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:16.143512 2075 hybrid_clock.cc:648] HybridClock initialized: now 1786518616143445 us; error 32 us; skew 500 ppm
I20260812 07:10:16.143822 2075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:16.151839 2075 init.cc:377] Logged in from keytab as oryx/127.21.181.125@KRBTEST.COM (short username oryx)
I20260812 07:10:16.153450 2075 webserver.cc:460] Webserver started at http://127.21.181.125:32965/ using document root <none> and password file <none>
I20260812 07:10:16.154155 2075 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:16.154258 2075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:16.154592 2075 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:16.157629 2075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/instance:
uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "6c47f265576df85bbd639719aa6ad08c"
server_key_iv: "c60c4986c69e1030aa2354385d25131f"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.158422 2075 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal/instance:
uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "6c47f265576df85bbd639719aa6ad08c"
server_key_iv: "c60c4986c69e1030aa2354385d25131f"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.163928 2075 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 07:10:16.167286 2105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:16.168603 2075 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:16.168771 2075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "6c47f265576df85bbd639719aa6ad08c"
server_key_iv: "c60c4986c69e1030aa2354385d25131f"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.168967 2075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:16.182092 2075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:16.185982 2075 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:16.186183 2075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:16.203433 2075 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.125:37625
I20260812 07:10:16.203460 2214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.125:37625 every 8 connection(s)
I20260812 07:10:16.204725 2075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
I20260812 07:10:16.207983 2218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:16.212026 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 2075
I20260812 07:10:16.214550 2218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:10:16.225513 1809 heartbeater.cc:503] Master 127.21.181.126:41905 was elected leader, sending a full tablet report...
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.125@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:16.235719 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.218611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.21.181.126:41905)
0812 07:10:16.218756 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.218760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.219539 (+ 779us) server_negotiation.cc:407] Connection header received
0812 07:10:16.219736 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.219740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.219824 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.219903 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.221403 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.223228 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.224815 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.225237 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.225343 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.228766 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.228771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.228774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.228812 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.232911 (+ 4099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.233733 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.233737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.233739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.233866 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.234237 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.234240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.234243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.234558 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.234686 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.235192 (+ 506us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.235382 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:10:16.236002 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.218681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:42448)
0812 07:10:16.219364 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.219419 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.219615 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.219986 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.219993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.220825 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.221205 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.221224 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.223373 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.223377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.224677 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.224690 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.225451 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.225459 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.225605 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.226171 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.226199 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.228594 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.233076 (+ 4482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.233087 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.233154 (+ 67us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.233611 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.233974 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.233979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.233983 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.234128 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.234690 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.234698 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.234846 (+ 148us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.235063 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.235366 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":2,"thread_start_us":324,"threads_started":1}
I20260812 07:10:16.236033 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.214069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.21.181.125:37625)
0812 07:10:16.214878 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.214892 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.214939 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:10:16.216700 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.216727 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.217183 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.217566 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.219235 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.221226 (+ 1991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.222906 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.223538 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.223719 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.228124 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.228135 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.228151 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.228184 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.231304 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.232228 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.232236 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.232244 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.232438 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.233256 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.233262 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.233265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.233765 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.234056 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.234503 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.234929 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":571,"thread_start_us":343,"threads_started":1}
I20260812 07:10:16.238466 2218 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.125@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
I20260812 07:10:16.249096 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.239067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:37296)
0812 07:10:16.239246 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.239264 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.239345 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.239559 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.239562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.239853 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.239990 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.239997 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.240952 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.240956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.241799 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.241810 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.242401 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.242405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.242561 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.243033 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.243048 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.244815 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.247157 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.247160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.247162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.247498 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.247791 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.247794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.247796 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.247919 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.248410 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.248413 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.248523 (+ 110us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.248623 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.248805 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:10:16.249179 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.239215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.21.181.125:37625)
0812 07:10:16.239343 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.239347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.239361 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:10:16.239403 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.239407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.239453 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.239541 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.240121 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.240826 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.241909 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.242290 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.242378 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.244982 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.244986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.244989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.245016 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.247046 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.247588 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.247591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.247593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.247706 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.247999 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.248002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.248004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.248306 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.248412 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.248687 (+ 275us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.248910 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":45}
I20260812 07:10:16.253589 2218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Bootstrap starting.
I20260812 07:10:16.256536 2218 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:16.257503 2218 log.cc:824] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:16.259843 2218 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: No bootstrap required, opened a new log
I20260812 07:10:16.266806 2218 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } }
I20260812 07:10:16.267083 2218 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:16.267174 2218 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f81a580229a4d7e8f92cbe0bc2d7bab, State: Initialized, Role: FOLLOWER
I20260812 07:10:16.267671 2218 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } }
I20260812 07:10:16.268428 2229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } } }
I20260812 07:10:16.268729 2229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:16.269313 2218 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:10:16.277559 2248 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:10:16.277693 2248 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:10:16.279384 2075 master_runner.cc:428] Detected that this master 4f81a580229a4d7e8f92cbe0bc2d7bab is joining an existing cluster
I20260812 07:10:16.279471 2075 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.181.125:37625
I20260812 07:10:16.297255 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.283440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42462 (local address 127.21.181.126:41905)
0812 07:10:16.283815 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.283820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.283838 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:16.284044 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.284049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.284152 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.284236 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.285272 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.285984 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.287313 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.287726 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.287831 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.290622 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.290629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.290633 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.290674 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.294081 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.294848 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.294853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.294857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.295016 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.295567 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.295573 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.295576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.295910 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.296028 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.296690 (+ 662us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.296909 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260812 07:10:16.298440 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.283087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:42462)
0812 07:10:16.283735 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.283753 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.283873 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.284271 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.284276 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.284810 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.285138 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.285149 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.286116 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.286122 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.287183 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.287195 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.287898 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.287905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.288074 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.288792 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.288814 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.290394 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.294226 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.294231 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.294234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.294708 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.295166 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.295171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.295175 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.295411 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.296150 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.296155 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.296367 (+ 212us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.297600 (+ 1233us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.297931 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"spinlock_wait_cycles":20736,"thread_start_us":755,"threads_started":1}
I20260812 07:10:16.303134 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.283082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.21.181.125:37625)
0812 07:10:16.283677 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.283683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.288591 (+ 4908us) server_negotiation.cc:407] Connection header received
0812 07:10:16.288740 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.288745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.288809 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.288871 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.290612 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.291626 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.293505 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.294211 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.294291 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.296541 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.296547 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.296552 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.296583 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.300257 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.301043 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.301048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.301052 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.301194 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.301601 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.301606 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.301609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.301950 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.302121 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.302587 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.302792 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439}
I20260812 07:10:16.303979 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.283351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:37300)
0812 07:10:16.288501 (+ 5150us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.288518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.288621 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.289066 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.289070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.290121 (+ 1051us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.290493 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.290505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.291841 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.291846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.293360 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.293379 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.294573 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.294580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.294748 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.295321 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.295342 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.296362 (+ 1020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.300388 (+ 4026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.300393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.300397 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.300918 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.301316 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.301320 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.301323 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.301475 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.302192 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.302196 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.302337 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.303480 (+ 1143us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.303697 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5048,"thread_start_us":4870,"threads_started":1}
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.126@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
I20260812 07:10:16.318828 2222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.305019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.21.181.125:37625)
0812 07:10:16.305194 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.305200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.305951 (+ 751us) server_negotiation.cc:407] Connection header received
0812 07:10:16.306260 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.306264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.306359 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.306436 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.307804 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.308530 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.309770 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.310230 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.310332 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.313300 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.313304 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.313307 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.313335 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.315841 (+ 2506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.316674 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.316681 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.316690 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.316851 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.317243 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.317248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.317250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.317534 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.317697 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.318189 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.318410 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:10:16.318879 2262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.305138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:37304)
0812 07:10:16.305820 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.305871 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.306078 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.306516 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.306523 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.307339 (+ 816us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.307640 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.307653 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.308657 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.308662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.309622 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.309634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.310374 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.310380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.310565 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.311078 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.311100 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.313129 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.316046 (+ 2917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.316061 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.316069 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.316546 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.316972 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.316978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.316982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.317128 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.317691 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.317700 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.317890 (+ 190us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.318063 (+ 173us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.318409 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":271,"threads_started":1}
I20260812 07:10:16.321372 1183 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.181.125:37625: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
member_type: NON_VOTER
last_known_addr {
host: "127.21.181.125"
port: 37625
}
attrs {
promote: true
}
}
dest_uuid: "ad16eee3ad2d49539a1039a65084d1ac"
cas_config_opid_index: -1
I20260812 07:10:16.322500 1183 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } }
WARNING: no policy specified for oryx/127.21.181.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.124@KRBTEST.COM" created.
I20260812 07:10:16.338462 2163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:16.338173 2267 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:16.342221 2267 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.181.125:37625
I20260812 07:10:16.342622 2163 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260812 07:10:16.343086 2075 master.cc:537] Master@127.21.181.125:37625 shutting down...
I20260812 07:10:16.348786 2266 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:16.349179 2266 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:16.349840 2266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:10:16.358942 1159 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260812 07:10:16.360797 1159 consensus_peers.cc:487] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac -> Peer 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125:37625): Couldn't send request to peer 4f81a580229a4d7e8f92cbe0bc2d7bab. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.124.keytab.
Entry for principal oryx/127.21.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.124.keytab.
I20260812 07:10:16.365085 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:46377
--webserver_interface=127.21.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.124.keytab
--principal=oryx/127.21.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:10:16.381870 2075 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:16.382306 2075 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:16.382535 2075 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: stopping tablet replica
I20260812 07:10:16.388598 2075 master.cc:552] Master@127.21.181.125:37625 shutdown complete.
I20260812 07:10:16.388670 2075 master_runner.cc:305] Clearing existing system tablet
I20260812 07:10:16.389050 2075 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:16.389138 2075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:16.391311 2075 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260812 07:10:16.392751 2277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:16.393039 2075 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:16.393299 2075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 07:10:16.393414 2075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "6c47f265576df85bbd639719aa6ad08c"
server_key_iv: "c60c4986c69e1030aa2354385d25131f"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.396859 2075 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:16.401510 2075 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 07:10:16.401669 2075 log.cc:1196] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260812 07:10:16.402055 2075 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Deleting consensus metadata
I20260812 07:10:16.402274 2075 master_runner.cc:315] Copying system tablet from 127.21.181.126:41905
I20260812 07:10:16.410657 2075 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: tablet copy: Beginning tablet copy session from remote peer at address 127.21.181.126:41905
I20260812 07:10:16.423450 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.411347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.21.181.126:41905)
0812 07:10:16.411532 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.411538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.412576 (+ 1038us) server_negotiation.cc:407] Connection header received
0812 07:10:16.412762 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.412768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.412848 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.412932 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.413794 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.414583 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.415786 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.416255 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.416360 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.418108 (+ 1748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.418113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.418116 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.418148 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.420674 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.421439 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.421445 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.421448 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.421579 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.421942 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.421947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.421951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.422363 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.422510 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.422909 (+ 399us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.423121 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:10:16.423527 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.411328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:42464)
0812 07:10:16.412462 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.412480 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.412635 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.412985 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.412999 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.413402 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.413642 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.413651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.414716 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.414722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.415645 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.415657 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.416409 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.416414 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.416592 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.417155 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.417173 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.417905 (+ 732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.420797 (+ 2892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.420802 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.420807 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.421333 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.421689 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.421694 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.421698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.421842 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.422491 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.422496 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.422659 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.422782 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.423093 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1000,"thread_start_us":881,"threads_started":1}
I20260812 07:10:16.424593 1218 tablet_copy_service.cc:140] P ad16eee3ad2d49539a1039a65084d1ac: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4f81a580229a4d7e8f92cbe0bc2d7bab ({username='oryx', principal='oryx/127.21.181.125@KRBTEST.COM'} at 127.0.0.1:42464)
I20260812 07:10:16.424852 1218 tablet_copy_service.cc:161] P ad16eee3ad2d49539a1039a65084d1ac: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4f81a580229a4d7e8f92cbe0bc2d7bab at {username='oryx', principal='oryx/127.21.181.125@KRBTEST.COM'} at 127.0.0.1:42464: session id = 4f81a580229a4d7e8f92cbe0bc2d7bab-00000000000000000000000000000000
I20260812 07:10:16.427752 1218 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 07:10:16.429512 2075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 07:10:16.434290 2075 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: tablet copy: Starting download of 0 data blocks...
I20260812 07:10:16.434559 2075 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: tablet copy: Starting download of 1 WAL segments...
I20260812 07:10:16.436893 2075 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 07:10:16.439978 1218 tablet_copy_service.cc:342] P ad16eee3ad2d49539a1039a65084d1ac: Request end of tablet copy session 4f81a580229a4d7e8f92cbe0bc2d7bab-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.181.125@KRBTEST.COM'} at 127.0.0.1:42464
I20260812 07:10:16.440168 1218 tablet_copy_service.cc:433] P ad16eee3ad2d49539a1039a65084d1ac: ending tablet copy session 4f81a580229a4d7e8f92cbe0bc2d7bab-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4f81a580229a4d7e8f92cbe0bc2d7bab
I20260812 07:10:16.449802 2075 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:16.455276 2290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:16.468214 2292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:16.469250 2075 server_base.cc:1034] running on GCE node
W20260812 07:10:16.469611 2299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:16.470067 2075 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:16.470693 2075 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:16.471940 2075 hybrid_clock.cc:648] HybridClock initialized: now 1786518616471879 us; error 40 us; skew 500 ppm
I20260812 07:10:16.472200 2075 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:16.473946 2075 init.cc:377] Logged in from keytab as oryx/127.21.181.125@KRBTEST.COM (short username oryx)
I20260812 07:10:16.475035 2075 webserver.cc:460] Webserver started at http://127.21.181.125:41765/ using document root <none> and password file <none>
I20260812 07:10:16.475414 2075 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:16.475504 2075 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:16.477957 2075 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260812 07:10:16.479526 2318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:16.479962 2075 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260812 07:10:16.480118 2075 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "6c47f265576df85bbd639719aa6ad08c"
server_key_iv: "c60c4986c69e1030aa2354385d25131f"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.480268 2075 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:16.488157 2075 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:16.491163 2075 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:16.532689 2075 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.125:37625
I20260812 07:10:16.532832 2418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.125:37625 every 8 connection(s)
I20260812 07:10:16.533663 2075 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
I20260812 07:10:16.539364 2419 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:10:16.542394 2419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Bootstrap starting.
I20260812 07:10:16.554904 2419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:10:16.555352 2419 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: Bootstrap complete.
I20260812 07:10:16.556186 2419 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } }
I20260812 07:10:16.556557 2419 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4f81a580229a4d7e8f92cbe0bc2d7bab, State: Initialized, Role: LEARNER
I20260812 07:10:16.556792 2419 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } }
I20260812 07:10:16.557552 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:16.557806 2419 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:16.557909 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: LEARNER
I20260812 07:10:16.569442 2448 catalog_manager.cc:1259] Loaded cluster ID: 82424b418be74224ace6b4218c4e4b12
I20260812 07:10:16.569531 2448 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: loading cluster ID for follower catalog manager: success
I20260812 07:10:16.571777 2448 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: acquiring CA information for follower catalog manager: success
I20260812 07:10:16.573625 2448 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:10:16.738265 2068 heartbeater.cc:503] Master 127.21.181.126:41905 was elected leader, sending a full tablet report...
W20260812 07:10:16.775594 2274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:16.775900 2274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:16.776022 2274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:16.783277 2274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:16.783382 2274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:16.783442 2274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:16.783504 2274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:16.783553 2274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:16.793191 2274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.124.keytab
--principal=oryx/127.21.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:46377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:16.794468 2274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:16.795697 2274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:16.806926 2465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:16.806972 2477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:16.807009 2467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:16.807684 2274 server_base.cc:1034] running on GCE node
I20260812 07:10:16.808506 2274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:16.809815 2274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:16.811025 2274 hybrid_clock.cc:648] HybridClock initialized: now 1786518616810967 us; error 50 us; skew 500 ppm
I20260812 07:10:16.811357 2274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:16.814358 2274 init.cc:377] Logged in from keytab as oryx/127.21.181.124@KRBTEST.COM (short username oryx)
I20260812 07:10:16.815884 2274 webserver.cc:460] Webserver started at http://127.21.181.124:43443/ using document root <none> and password file <none>
I20260812 07:10:16.816577 2274 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:16.816710 2274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:16.817027 2274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:16.820142 2274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/instance:
uuid: "176509da617f4f9f95ed259d9f9766a1"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "4786a9f04ce0653aec82ebb005f564d9"
server_key_iv: "6570285e3f648f6e93a7d39a1e495338"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.820979 2274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal/instance:
uuid: "176509da617f4f9f95ed259d9f9766a1"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "4786a9f04ce0653aec82ebb005f564d9"
server_key_iv: "6570285e3f648f6e93a7d39a1e495338"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.826138 2274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:10:16.829537 2490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:16.830777 2274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:10:16.830961 2274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
uuid: "176509da617f4f9f95ed259d9f9766a1"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "4786a9f04ce0653aec82ebb005f564d9"
server_key_iv: "6570285e3f648f6e93a7d39a1e495338"
server_key_version: "encryptionkey@0"
I20260812 07:10:16.831182 2274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:16.851529 2274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:16.855263 2274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:16.855530 2274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:16.874106 2274 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.124:46377
I20260812 07:10:16.874109 2602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.124:46377 every 8 connection(s)
I20260812 07:10:16.875531 2274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
I20260812 07:10:16.879038 2605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:16.881166 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 2274
I20260812 07:10:16.887593 2605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.124@KRBTEST.COM
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.124@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:16.903632 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.883352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54382 (local address 127.21.181.124:46377)
0812 07:10:16.884164 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.884175 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.884215 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:10:16.885747 (+ 1532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.885772 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.886129 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.886454 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.887899 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.889236 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.890851 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.891629 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.891751 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.896447 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.896457 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.896474 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.896531 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.899659 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.900323 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.900332 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.900339 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.900519 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.900853 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.900857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.900860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.901484 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.901799 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.902096 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.902451 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":348,"threads_started":1}
I20260812 07:10:16.904918 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.890038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.21.181.126:41905)
0812 07:10:16.890226 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.890232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.890662 (+ 430us) server_negotiation.cc:407] Connection header received
0812 07:10:16.890830 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.890835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.890916 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.891031 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.892210 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.893013 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.894903 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.895280 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.895387 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.898940 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.898947 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.898952 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.899001 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.901940 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.902811 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.902817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.902821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.902980 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.903355 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.903360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.903363 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.903711 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.903852 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.904510 (+ 658us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.904670 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:10:16.905081 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.889820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:42480)
0812 07:10:16.890512 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.890556 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.890714 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.891079 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.891083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.891702 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.892051 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.892064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.893178 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.893183 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.894774 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.894790 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.895496 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.895504 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.895694 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.896232 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.896254 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.898755 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.902079 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.902090 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.902096 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.902708 (+ 612us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.903107 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.903110 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.903112 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.903254 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.903923 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.903930 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.904173 (+ 243us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.904390 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.904675 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":329,"threads_started":1}
I20260812 07:10:16.906440 2605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.124@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
I20260812 07:10:16.917927 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.906967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:37314)
0812 07:10:16.907133 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.907147 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.907219 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.907895 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.907898 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.908222 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.908353 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.908361 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.909534 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.909538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.910408 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.910418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.911006 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.911010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.911142 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.911591 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.911606 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.913423 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.916135 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.916140 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.916144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.916502 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.916800 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.916803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.916805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.916924 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.917312 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.917314 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.917417 (+ 103us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.917527 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.917715 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:10:16.917976 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.907137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.21.181.125:37625)
0812 07:10:16.907664 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.907668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.907683 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:10:16.907732 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.907735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.907791 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.907876 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.908467 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.909432 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.910517 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.910905 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.911001 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.913586 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.913590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.913593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.913626 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.916015 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.916590 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.916594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.916596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.916725 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.917004 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.917006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.917008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.917236 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.917349 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.917591 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.917735 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":294,"threads_started":1}
I20260812 07:10:16.918906 2605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518616, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.124@KRBTEST.COM for oryx/127.21.181.124@KRBTEST.COM
I20260812 07:10:16.930138 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.919436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.0.0.1:54396)
0812 07:10:16.919591 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.919608 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.919701 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.919984 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.919987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.920320 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.920458 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.920465 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.921477 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.921482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.922416 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.922429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.923052 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.923056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.923223 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.923650 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.923665 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.925480 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.927865 (+ 2385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.927870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.927874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.928356 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.928668 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.928672 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.928675 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.928811 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.929273 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.929278 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.929405 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.929508 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.929734 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:10:16.930166 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.919655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.21.181.124:46377)
0812 07:10:16.919782 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.919786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.919799 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:10:16.919833 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.919836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.919884 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.919952 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.920573 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.921367 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.922531 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.922914 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.923016 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.925650 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.925654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.925657 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.925684 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.927752 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.928459 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.928464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.928466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.928575 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.928915 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.928919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.928920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.929170 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.929292 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.929623 (+ 331us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.929874 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":45}
I20260812 07:10:16.934751 2605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Bootstrap starting.
I20260812 07:10:16.937975 2605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:16.938944 2605 log.cc:824] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:16.941454 2605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: No bootstrap required, opened a new log
I20260812 07:10:16.948282 2605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } }
I20260812 07:10:16.948586 2605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:16.948698 2605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176509da617f4f9f95ed259d9f9766a1, State: Initialized, Role: FOLLOWER
I20260812 07:10:16.949255 2605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } }
I20260812 07:10:16.950066 2618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } } }
I20260812 07:10:16.950351 2618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:16.951002 2605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:16.958372 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 1301
I20260812 07:10:16.960906 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.953864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:37324)
0812 07:10:16.954464 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.954480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.954618 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.954987 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.954992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.955021 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:16.955215 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.955224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.956401 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.956409 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.958359 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.958377 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.960055 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.960061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.960283 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.960341 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.960438 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.960569 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":374,"threads_started":1}
I20260812 07:10:16.961563 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.954129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.21.181.125:37625)
0812 07:10:16.954327 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.954334 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.954574 (+ 240us) server_negotiation.cc:407] Connection header received
0812 07:10:16.954743 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.954749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.954842 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.954932 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:16.955347 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.956246 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.958537 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.959754 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.959921 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.960131 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.961182 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.961333 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260812 07:10:16.962957 2362 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260812 07:10:16.963524 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad16eee3ad2d49539a1039a65084d1ac. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:16.963797 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: LEARNER
W20260812 07:10:16.964704 2647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:10:16.964890 2647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:10:16.968000 2274 master_runner.cc:428] Detected that this master 176509da617f4f9f95ed259d9f9766a1 is joining an existing cluster
I20260812 07:10:16.968142 2274 master_runner.cc:432] Initiating AddMaster RPC to add 127.21.181.124:46377
Aug 12 07:10:16 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.65@KRBTEST.COM
I20260812 07:10:16.983425 2652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.959764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42308 (local address 127.21.181.65:44169)
0812 07:10:16.963215 (+ 3451us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.963223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.963250 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:10:16.963315 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.963323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.963575 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.965393 (+ 1818us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.965607 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.966886 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.968317 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.969157 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.970105 (+ 948us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.974916 (+ 4811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.974930 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.974944 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.974999 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.978980 (+ 3981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.980050 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.980058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.980062 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.980236 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.980737 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.980743 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.980746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.981194 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.981480 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.982758 (+ 1278us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.983028 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3301,"thread_start_us":3158,"threads_started":1}
I20260812 07:10:16.985036 1526 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:16.993079 2664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.975783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.0.0.1:54402)
0812 07:10:16.976376 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.976391 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.976540 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.977004 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.977008 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.977899 (+ 891us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.978325 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.978339 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.979843 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.979848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.981815 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.981832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.983158 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.983165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.983352 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.984012 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.984058 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.986102 (+ 2044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:16.989949 (+ 3847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.989954 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.989958 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.990470 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.990869 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:16.990873 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:16.990877 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:16.991055 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:16.991752 (+ 697us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:16.991760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:16.991951 (+ 191us) client_negotiation.cc:769] Sending connection context
0812 07:10:16.992171 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:10:16.992522 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"spinlock_wait_cycles":40320,"thread_start_us":510,"threads_started":1}
I20260812 07:10:16.994225 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.976101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.21.181.124:46377)
0812 07:10:16.976292 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.976297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.976454 (+ 157us) server_negotiation.cc:407] Connection header received
0812 07:10:16.976666 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.976671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.976738 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.976804 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.978478 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.979622 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.981963 (+ 2341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.982679 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.982793 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.986253 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.986259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.986266 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.986299 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:16.989831 (+ 3532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.990592 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.990597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.990601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.990759 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:16.991156 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:16.991162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:16.991165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:16.991512 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:16.991700 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:16.993599 (+ 1899us) server_negotiation.cc:299] Negotiation successful
0812 07:10:16.993823 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:10:17.003607 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.974915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42482 (local address 127.21.181.126:41905)
0812 07:10:16.975090 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.975096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.975244 (+ 148us) server_negotiation.cc:407] Connection header received
0812 07:10:16.975512 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.975518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.975593 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.975662 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.986702 (+ 11040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.987764 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.989893 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.990545 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.990671 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.997170 (+ 6499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:16.997177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:16.997183 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:16.997230 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.000291 (+ 3061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.001119 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.001125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.001129 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.001287 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.001757 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.001762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.001765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.002169 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.002298 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.002942 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.003178 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:10:17.004823 2661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.974485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:42482)
0812 07:10:16.975144 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.975163 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.975357 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.984702 (+ 9345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.984713 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.985483 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.985860 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.985873 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.987985 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.987990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.989720 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.989736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.990962 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.990969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.991152 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:16.991793 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:16.991815 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:16.996877 (+ 5062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.000445 (+ 3568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.000450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.000454 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.000981 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.001448 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.001452 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.001455 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.001640 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.002414 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.002418 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.002646 (+ 228us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.004003 (+ 1357us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.004363 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"spinlock_wait_cycles":21760,"thread_start_us":613,"threads_started":1}
I20260812 07:10:17.009173 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.976783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.21.181.125:37625)
0812 07:10:16.976936 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:16.976941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:16.984233 (+ 7292us) server_negotiation.cc:407] Connection header received
0812 07:10:16.984665 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:16.984670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:16.984747 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:16.984814 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:16.992980 (+ 8166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.993951 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.995970 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.996669 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.996768 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.001802 (+ 5034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.001808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.001813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.001853 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.005236 (+ 3383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.006355 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.006361 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.006364 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.006505 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.007147 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.007153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.007156 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.007471 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.007615 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.008605 (+ 990us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.008839 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:10:17.010764 2666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:16.977376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:37334)
0812 07:10:16.984144 (+ 6768us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:16.984160 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:16.984494 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:16.985027 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:16.985032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:16.992445 (+ 7413us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:16.992783 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.992795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.994152 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.994157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.995803 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:16.995819 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:16.998791 (+ 2972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:16.998798 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:16.998962 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.000302 (+ 1340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:17.000323 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:17.001593 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.005750 (+ 4157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.005754 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.005757 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.006206 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.006764 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.006768 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.006771 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.007013 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.007693 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.007698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.008277 (+ 579us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.009610 (+ 1333us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.009940 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6619,"thread_start_us":6868,"threads_started":1}
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.126@KRBTEST.COM for oryx/127.21.181.124@KRBTEST.COM
I20260812 07:10:17.022466 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.010588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.0.0.1:54404)
0812 07:10:17.010772 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:17.010791 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:17.010895 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:17.011311 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:17.011316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:17.011739 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:17.011910 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.011921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.013038 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.013044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.013893 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.013904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.014648 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.014654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.014825 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.015306 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:17.015322 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:17.017331 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.020242 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.020247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.020251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.020607 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.020938 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.020943 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.020947 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.021102 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.021587 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.021592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.021745 (+ 153us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.021861 (+ 116us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.022136 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:10:17.022470 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.010736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54404 (local address 127.21.181.124:46377)
0812 07:10:17.010908 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.010913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.010962 (+ 49us) server_negotiation.cc:407] Connection header received
0812 07:10:17.011065 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.011071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.011173 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.011271 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:17.012126 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.012904 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.014035 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.014515 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.014625 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.017503 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.017508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.017511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.017543 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.020122 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.020720 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.020725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.020728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.020836 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.021206 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.021210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.021214 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.021466 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.021600 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.021972 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.022154 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:17.024032 1183 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.21.181.124:46377: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "176509da617f4f9f95ed259d9f9766a1"
member_type: NON_VOTER
last_known_addr {
host: "127.21.181.124"
port: 46377
}
attrs {
promote: true
}
}
dest_uuid: "ad16eee3ad2d49539a1039a65084d1ac"
cas_config_opid_index: 5
I20260812 07:10:17.024992 1183 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } }
I20260812 07:10:17.026536 2626 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.027144 2362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 LEARNER]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 07:10:17.029201 2556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:17.033151 2556 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260812 07:10:17.064940 2673 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 07:10:17.076151 2362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.080662 2673 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:17.092206 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.092540 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: LEARNER
I20260812 07:10:17.103340 2556 raft_consensus.cc:858] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } }
W20260812 07:10:17.103845 2556 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } }
I20260812 07:10:17.106755 2626 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.21.181.124:46377
W20260812 07:10:17.107467 2499 tablet.cc:2369] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:17.107765 2671 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.108330 2671 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.108877 2626 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: attempting to promote NON_VOTER 4f81a580229a4d7e8f92cbe0bc2d7bab to VOTER
I20260812 07:10:17.110077 2626 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } }
I20260812 07:10:17.113579 2673 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.114030 2673 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.116029 2362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 LEARNER]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 07:10:17.116854 2673 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260812 07:10:17.120187 2362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.121423 2671 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.125051 2425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.125298 2425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:17.128774 2684 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.128806 2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.129231 2684 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.129284 2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.133805 2556 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:17.137368 2274 master.cc:537] Master@127.21.181.124:46377 shutting down...
I20260812 07:10:17.143105 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 1301
I20260812 07:10:17.160064 2553 raft_consensus.cc:1467] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Allowing update even though not a member of the config
I20260812 07:10:17.160341 2553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260812 07:10:17.160612 2618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad16eee3ad2d49539a1039a65084d1ac. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:17.160864 2618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 07:10:17.161131 2682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:17.161414 2682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 07:10:17.159736 2691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: NON_VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: true } } }
I20260812 07:10:17.161769 2691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260812 07:10:17.161811 2670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260812 07:10:17.165328 2274 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260812 07:10:17.165902 2274 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260812 07:10:17.166147 2274 tablet_replica.cc:330] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: stopping tablet replica
W20260812 07:10:17.173408 1153 consensus_peers.cc:487] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac -> Peer 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124:46377): Couldn't send request to peer 176509da617f4f9f95ed259d9f9766a1. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260812 07:10:17.175908 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:44169
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:41905
--webserver_port=44515
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:10:17.185082 2274 master.cc:552] Master@127.21.181.124:46377 shutdown complete.
I20260812 07:10:17.185177 2274 master_runner.cc:305] Clearing existing system tablet
I20260812 07:10:17.185606 2274 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:17.185735 2274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:17.188944 2274 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260812 07:10:17.190778 2705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:17.191143 2274 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:17.191419 2274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260812 07:10:17.191532 2274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
uuid: "176509da617f4f9f95ed259d9f9766a1"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "4786a9f04ce0653aec82ebb005f564d9"
server_key_iv: "6570285e3f648f6e93a7d39a1e495338"
server_key_version: "encryptionkey@0"
I20260812 07:10:17.196429 2274 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:17.203295 2274 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260812 07:10:17.203450 2274 log.cc:1196] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260812 07:10:17.203970 2274 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Deleting consensus metadata
I20260812 07:10:17.204305 2274 master_runner.cc:315] Copying system tablet from 127.21.181.126:41905
I20260812 07:10:17.209640 2274 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: tablet copy: Beginning tablet copy session from remote peer at address 127.21.181.126:41905
I20260812 07:10:17.225337 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.210475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.21.181.126:41905)
0812 07:10:17.210723 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.210729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.211248 (+ 519us) server_negotiation.cc:407] Connection header received
0812 07:10:17.211427 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.211432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.211517 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.211625 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:17.212731 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.213887 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.215742 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.216414 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.216542 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.218774 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.218780 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.218786 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.218825 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.222264 (+ 3439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.223076 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.223082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.223085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.223255 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.223720 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.223725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.223728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.224149 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.224294 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.224732 (+ 438us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.224971 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260812 07:10:17.225395 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.210434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:42496)
0812 07:10:17.211119 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:17.211139 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:17.211298 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:17.211678 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:17.211683 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:17.212235 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:17.212562 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.212581 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.214061 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.214067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.215529 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.215555 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.216585 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.216591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.216782 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.217452 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:17.217477 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:17.218556 (+ 1079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.222400 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.222406 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.222410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.222954 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.223383 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.223408 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.223411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.223601 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.224268 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.224273 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.224466 (+ 193us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.224617 (+ 151us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.224961 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":423,"threads_started":1}
I20260812 07:10:17.226330 1218 tablet_copy_service.cc:140] P ad16eee3ad2d49539a1039a65084d1ac: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 176509da617f4f9f95ed259d9f9766a1 ({username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:42496)
I20260812 07:10:17.226521 1218 tablet_copy_service.cc:161] P ad16eee3ad2d49539a1039a65084d1ac: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 176509da617f4f9f95ed259d9f9766a1 at {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:42496: session id = 176509da617f4f9f95ed259d9f9766a1-00000000000000000000000000000000
I20260812 07:10:17.230050 1218 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Tablet Copy: opened 0 blocks and 1 log segments
I20260812 07:10:17.231807 2274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260812 07:10:17.238745 2274 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: tablet copy: Starting download of 0 data blocks...
I20260812 07:10:17.239039 2274 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: tablet copy: Starting download of 1 WAL segments...
I20260812 07:10:17.241253 2274 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 07:10:17.245487 1218 tablet_copy_service.cc:342] P ad16eee3ad2d49539a1039a65084d1ac: Request end of tablet copy session 176509da617f4f9f95ed259d9f9766a1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:42496
I20260812 07:10:17.245622 1218 tablet_copy_service.cc:433] P ad16eee3ad2d49539a1039a65084d1ac: ending tablet copy session 176509da617f4f9f95ed259d9f9766a1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 176509da617f4f9f95ed259d9f9766a1
I20260812 07:10:17.256206 2274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:17.258466 2719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:17.264499 2720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:17.268996 2723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:17.269232 2274 server_base.cc:1034] running on GCE node
I20260812 07:10:17.269704 2274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:17.270287 2274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:17.271528 2274 hybrid_clock.cc:648] HybridClock initialized: now 1786518617271485 us; error 33 us; skew 500 ppm
I20260812 07:10:17.271741 2274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518617, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:17.273516 2274 init.cc:377] Logged in from keytab as oryx/127.21.181.124@KRBTEST.COM (short username oryx)
I20260812 07:10:17.274782 2274 webserver.cc:460] Webserver started at http://127.21.181.124:44633/ using document root <none> and password file <none>
I20260812 07:10:17.275190 2274 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:17.275298 2274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:17.277748 2274 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:10:17.279285 2737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:17.279672 2274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260812 07:10:17.279784 2274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
uuid: "176509da617f4f9f95ed259d9f9766a1"
format_stamp: "Formatted at 2026-08-12 07:10:16 on dist-test-slave-lgr5"
server_key: "4786a9f04ce0653aec82ebb005f564d9"
server_key_iv: "6570285e3f648f6e93a7d39a1e495338"
server_key_version: "encryptionkey@0"
I20260812 07:10:17.279968 2274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:17.300398 2274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:17.303659 2274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:17.331625 2274 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.124:46377
I20260812 07:10:17.331710 2835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.124:46377 every 8 connection(s)
I20260812 07:10:17.332659 2274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
I20260812 07:10:17.338392 2837 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:10:17.342751 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Bootstrap starting.
I20260812 07:10:17.354198 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:10:17.354593 2837 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Bootstrap complete.
I20260812 07:10:17.355413 2837 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } }
I20260812 07:10:17.355638 2837 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 176509da617f4f9f95ed259d9f9766a1, State: Initialized, Role: LEARNER
I20260812 07:10:17.355898 2837 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } }
I20260812 07:10:17.358151 2837 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:17.357831 2843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.358421 2843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:10:17.376680 2862 catalog_manager.cc:1259] Loaded cluster ID: 82424b418be74224ace6b4218c4e4b12
I20260812 07:10:17.376760 2862 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: loading cluster ID for follower catalog manager: success
I20260812 07:10:17.379158 2862 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: acquiring CA information for follower catalog manager: success
I20260812 07:10:17.381311 2862 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:10:17.574519 2699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:17.574788 2699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:17.574926 2699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:17.582440 2699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:17.582568 2699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:17.582727 2699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:17.592509 2699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:44169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=44515
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:17.593734 2699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:17.594983 2699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:17.607752 2874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:17.607803 2871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:17.607833 2869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:17.608717 2699 server_base.cc:1034] running on GCE node
I20260812 07:10:17.609256 2699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:17.610085 2699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:17.611284 2699 hybrid_clock.cc:648] HybridClock initialized: now 1786518617611242 us; error 27 us; skew 500 ppm
I20260812 07:10:17.611572 2699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518617, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:17.614554 2699 init.cc:377] Logged in from keytab as oryx/127.21.181.65@KRBTEST.COM (short username oryx)
I20260812 07:10:17.615898 2699 webserver.cc:460] Webserver started at http://127.21.181.65:44515/ using document root <none> and password file <none>
I20260812 07:10:17.616564 2699 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:17.616698 2699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:17.621727 2699 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:10:17.624261 2887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:17.625458 2699 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:10:17.625641 2699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "c883c5578a264138870f8efc4d129bd2"
format_stamp: "Formatted at 2026-08-12 07:10:14 on dist-test-slave-lgr5"
server_key: "2dc708a1daa44afe30c6a20a6ee15ff3"
server_key_iv: "625c98449a29183a5b0e3af750d57077"
server_key_version: "encryptionkey@0"
I20260812 07:10:17.626672 2699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:17.642699 2699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:17.646674 2699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:17.646894 2699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:17.647902 2699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:17.649245 2699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:17.649330 2699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:17.649410 2699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:17.649464 2699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:17.680559 2699 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:44169
I20260812 07:10:17.680627 3127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:44169 every 8 connection(s)
I20260812 07:10:17.682071 2699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:17.683310 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 2699
I20260812 07:10:17.683643 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 1544
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.66@KRBTEST.COM
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518617, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518617, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.124@KRBTEST.COM
Aug 12 07:10:17 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518617, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:17.724027 3144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.693513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:47343 (local address 127.21.181.125:37625)
0812 07:10:17.695521 (+ 2008us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.695528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.702143 (+ 6615us) server_negotiation.cc:407] Connection header received
0812 07:10:17.704911 (+ 2768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.704917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.705014 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.705123 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:17.707657 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.708757 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.711484 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.712126 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.712230 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.716926 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.716934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.716939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.716977 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.720401 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.721280 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.721285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.721288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.721428 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.721963 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.721968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.721971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.722331 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.722446 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.723415 (+ 969us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.723635 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1865,"thread_start_us":1742,"threads_started":1}
I20260812 07:10:17.724192 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.702285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:33361 (local address 127.21.181.124:46377)
0812 07:10:17.702848 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.702855 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.704616 (+ 1761us) server_negotiation.cc:407] Connection header received
0812 07:10:17.705362 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.705367 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.705453 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.705575 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:17.707657 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.709069 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.711439 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.712014 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.712138 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.718055 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.718060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.718063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.718096 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.721275 (+ 3179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.722051 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.722055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.722059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.722189 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.722639 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.722644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.722647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.722973 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.723108 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.723635 (+ 527us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.723832 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":224,"threads_started":1}
I20260812 07:10:17.725378 3135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.685014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.21.181.66:39441)
0812 07:10:17.700655 (+ 15641us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.700665 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.702047 (+ 1382us) server_negotiation.cc:407] Connection header received
0812 07:10:17.702280 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.702289 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.702547 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.702697 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:17.705062 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.706571 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.708380 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.709421 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.709542 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.716945 (+ 7403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.716959 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.716975 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.717016 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.721341 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.722204 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.722213 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.722216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.722366 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.722824 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.722829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.722832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.723265 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.723592 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.724718 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.724977 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15440,"thread_start_us":15240,"threads_started":1}
I20260812 07:10:17.726161 1537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.684755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:44141 (local address 127.21.181.126:41905)
0812 07:10:17.684923 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.684930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.701398 (+ 16468us) server_negotiation.cc:407] Connection header received
0812 07:10:17.705104 (+ 3706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.705109 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.705192 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.705278 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:17.707659 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.708860 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.714375 (+ 5515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.715002 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.715119 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.719455 (+ 4336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:17.719461 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:17.719465 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:17.719505 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:17.722697 (+ 3192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.723920 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.723925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.723929 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.724126 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:17.724563 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:17.724568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:17.724571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:17.724931 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:17.725080 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.725627 (+ 547us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.725847 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:10:17.726665 3138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.693040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.21.181.65:47343)
0812 07:10:17.702054 (+ 9014us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:17.702071 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:17.704617 (+ 2546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:17.705372 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:17.705382 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:17.706314 (+ 932us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:17.707467 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.707491 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.708958 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.708964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.711279 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.711303 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.712603 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.712610 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.712880 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.713726 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:17.713749 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:17.716630 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.720567 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.720575 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.720592 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.721139 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.721570 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.721575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.721578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.721821 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.722554 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.722561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.722972 (+ 411us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.724583 (+ 1611us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.725099 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8882,"mutex_wait_us":340,"spinlock_wait_cycles":7680,"thread_start_us":7785,"threads_started":1}
I20260812 07:10:17.726819 1799 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:17.727063 3142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.685355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:44141)
0812 07:10:17.701238 (+ 15883us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:17.701283 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:17.704960 (+ 3677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:17.705435 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:17.705439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:17.706311 (+ 872us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:17.707454 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.707481 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.712635 (+ 5154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.712640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.714203 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.714219 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.715440 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.715444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.715558 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.716012 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:17.716039 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:17.719238 (+ 3199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.722836 (+ 3598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.722840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.722844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.723784 (+ 940us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.724275 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.724280 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.724284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.724428 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.725174 (+ 746us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.725179 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.725333 (+ 154us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.726343 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.726628 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15588,"mutex_wait_us":180,"spinlock_wait_cycles":77568,"thread_start_us":16263,"threads_started":1}
I20260812 07:10:17.728262 3130 heartbeater.cc:347] Connected to a master server at 127.21.181.126:41905
I20260812 07:10:17.728567 3130 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:17.729275 3130 heartbeater.cc:511] Master 127.21.181.126:41905 requested a full tablet report, sending...
I20260812 07:10:17.730019 3139 heartbeater.cc:347] Connected to a master server at 127.21.181.125:37625
I20260812 07:10:17.730154 3139 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:17.730531 3139 heartbeater.cc:511] Master 127.21.181.125:37625 requested a full tablet report, sending...
I20260812 07:10:17.731106 1183 ts_manager.cc:194] Re-registered known tserver with Master: c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:17.732234 1183 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:44141
I20260812 07:10:17.732403 2346 ts_manager.cc:194] Registered new tserver with Master: c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:17.732936 3148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.702722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.21.181.65:33361)
0812 07:10:17.704521 (+ 1799us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:17.704535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:17.704656 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:17.705810 (+ 1154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:17.705814 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:17.706384 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:17.707455 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.707481 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.709252 (+ 1771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.709257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.711270 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.711285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.712497 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.712506 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.712888 (+ 382us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.713670 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:17.713705 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:17.717235 (+ 3530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:17.721422 (+ 4187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.721427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.721430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.721931 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.722336 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:17.722340 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:17.722343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:17.722521 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:17.723207 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:17.723211 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:17.723349 (+ 138us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.724541 (+ 1192us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.725218 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1698,"mutex_wait_us":504,"spinlock_wait_cycles":5760,"thread_start_us":1538,"threads_started":1}
I20260812 07:10:17.734293 3131 heartbeater.cc:347] Connected to a master server at 127.21.181.124:46377
I20260812 07:10:17.734436 3131 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:17.734875 3131 heartbeater.cc:511] Master 127.21.181.124:46377 requested a full tablet report, sending...
I20260812 07:10:17.736377 2768 ts_manager.cc:194] Registered new tserver with Master: c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:17.765501 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.760382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54420 (local address 127.21.181.124:46377)
0812 07:10:17.760569 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:17.760574 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:17.760791 (+ 217us) server_negotiation.cc:407] Connection header received
0812 07:10:17.760961 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:17.760965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:17.761060 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:17.761155 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:17.761510 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.762199 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.763659 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.764610 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.764765 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.764980 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:17.765107 (+ 127us) server_negotiation.cc:299] Negotiation successful
0812 07:10:17.765250 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:10:17.765662 3152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:17.760223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.0.0.1:54420)
0812 07:10:17.760694 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:17.760709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:17.760832 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:17.761188 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:17.761193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:17.761218 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:17.761380 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.761389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.762334 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.762340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.763513 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:17.763525 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:17.764765 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:17.764792 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:17.764968 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:17.765000 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:10:17.765087 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:10:17.765277 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":259,"threads_started":1}
I20260812 07:10:17.766566 2792 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260812 07:10:17.767001 2843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad16eee3ad2d49539a1039a65084d1ac. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: NON_VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: true } } }
I20260812 07:10:17.767271 2843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: LEARNER
I20260812 07:10:17.767987 3156 raft_consensus.cc:1050] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: attempting to promote NON_VOTER 176509da617f4f9f95ed259d9f9766a1 to VOTER
I20260812 07:10:17.768571 3156 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } }
I20260812 07:10:17.770402 2792 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 LEARNER]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 07:10:17.770401 2362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Refusing update from remote peer ad16eee3ad2d49539a1039a65084d1ac: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260812 07:10:17.770944 2670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 07:10:17.771426 2670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260812 07:10:17.773648 3156 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.775018 2792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.777181 2362 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.777395 3156 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.777652 3156 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.777647 2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.777912 2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.778183 2670 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.778402 2670 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:17.781587 2843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.781829 2843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:17.788772 3163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ad16eee3ad2d49539a1039a65084d1ac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:17.789168 3163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:17.830063 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 1544
I20260812 07:10:17.849329 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:39441
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:41905
--webserver_port=44703
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:18.176745 3165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:18.177004 3165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:18.177104 3165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:18.184480 3165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:18.184582 3165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:18.184700 3165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:18.194017 3165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:39441
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=44703
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:18.195170 3165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:18.196403 3165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:18.208880 3172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:18.208961 3173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:18.208925 3175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:18.209626 3165 server_base.cc:1034] running on GCE node
I20260812 07:10:18.210235 3165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:18.211107 3165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:18.212284 3165 hybrid_clock.cc:648] HybridClock initialized: now 1786518618212234 us; error 45 us; skew 500 ppm
I20260812 07:10:18.212572 3165 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:18.219398 3165 init.cc:377] Logged in from keytab as oryx/127.21.181.66@KRBTEST.COM (short username oryx)
I20260812 07:10:18.220762 3165 webserver.cc:460] Webserver started at http://127.21.181.66:44703/ using document root <none> and password file <none>
I20260812 07:10:18.221417 3165 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:18.221549 3165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:18.226805 3165 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 07:10:18.229528 3186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:18.230871 3165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:10:18.231042 3165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1050870805f441d287bd01e27acedd9a"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "72daff45d1e69427bcc0788abd157512"
server_key_iv: "2b5a877ee2fdcb2ce5e35912b423e1be"
server_key_version: "encryptionkey@0"
I20260812 07:10:18.232101 3165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:18.264815 3165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:18.268785 3165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:18.269011 3165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:18.269889 3165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:18.271166 3165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:18.271252 3165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:18.271334 3165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:18.271392 3165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:18.308100 3165 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:39441
I20260812 07:10:18.308169 3415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:39441 every 8 connection(s)
I20260812 07:10:18.309921 3165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:18.313402 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 3165
I20260812 07:10:18.313722 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 1815
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.67@KRBTEST.COM
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.124@KRBTEST.COM
I20260812 07:10:18.342384 3437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.315180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.21.181.67:38483)
0812 07:10:18.315761 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.315770 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.315801 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:10:18.317068 (+ 1267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.317077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.317284 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.317421 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.319951 (+ 2530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.321530 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.323917 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.324963 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.325104 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.332193 (+ 7089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.332207 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.332222 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.332268 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.337550 (+ 5282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.339035 (+ 1485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.339044 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.339048 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.339214 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.339619 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.339624 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.339628 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.340182 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.340452 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.341715 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.341991 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":352,"threads_started":1}
I20260812 07:10:18.343115 3438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.314545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.21.181.66:42101)
0812 07:10:18.316556 (+ 2011us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:18.316575 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:18.318999 (+ 2424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:18.319973 (+ 974us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:18.319977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:18.320934 (+ 957us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:18.326639 (+ 5705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.326656 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.328236 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.328240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.329216 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.329227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.330385 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.330392 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.330545 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.331113 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:18.331135 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:18.333578 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:18.337571 (+ 3993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.337581 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.337597 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.338269 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.338707 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.338711 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.338715 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.338928 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.340429 (+ 1501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:18.340437 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:18.340875 (+ 438us) client_negotiation.cc:769] Sending connection context
0812 07:10:18.341185 (+ 310us) client_negotiation.cc:240] Negotiation successful
0812 07:10:18.341706 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1874,"mutex_wait_us":62,"thread_start_us":10362,"threads_started":1}
I20260812 07:10:18.344350 2060 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:18.348340 3435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.315124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:42101 (local address 127.21.181.125:37625)
0812 07:10:18.316053 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.316059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.316665 (+ 606us) server_negotiation.cc:407] Connection header received
0812 07:10:18.319139 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.319156 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.319243 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.319391 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.326861 (+ 7470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.328040 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.329357 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.329967 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.330093 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.333777 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.333784 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.333789 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.333842 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.337373 (+ 3531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.338402 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.338407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.338411 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.338570 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.339756 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.339761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.339764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.340187 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.340346 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.344827 (+ 4481us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.347915 (+ 3088us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":811,"thread_start_us":682,"threads_started":1}
I20260812 07:10:18.349948 3428 heartbeater.cc:347] Connected to a master server at 127.21.181.125:37625
I20260812 07:10:18.350238 3428 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:18.350471 3430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.313229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:39773 (local address 127.21.181.126:41905)
0812 07:10:18.313933 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.313941 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.316060 (+ 2119us) server_negotiation.cc:407] Connection header received
0812 07:10:18.319266 (+ 3206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.319273 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.319367 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.319450 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.322166 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.323306 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.325651 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.326072 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.326192 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.343786 (+ 17594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.343793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.343797 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.343840 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.347529 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.348357 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.348362 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.348366 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.348538 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.349021 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.349026 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.349030 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.349300 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.349431 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.349881 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.350095 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":472,"thread_start_us":332,"threads_started":1}
I20260812 07:10:18.351157 3428 heartbeater.cc:511] Master 127.21.181.125:37625 requested a full tablet report, sending...
I20260812 07:10:18.351558 3432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.313786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:39773)
0812 07:10:18.314856 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:18.315949 (+ 1093us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:18.318729 (+ 2780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:18.319936 (+ 1207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:18.319949 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:18.320939 (+ 990us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:18.321942 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.321969 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.323514 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.323520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.325524 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.325540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.326463 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.326471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.326770 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.327392 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:18.327424 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:18.330548 (+ 3124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:18.347686 (+ 17138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.347692 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.347696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.348213 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.348680 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.348683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.348685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.348875 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.349427 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:18.349431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:18.349608 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 07:10:18.350716 (+ 1108us) client_negotiation.cc:240] Negotiation successful
0812 07:10:18.351021 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":47,"spinlock_wait_cycles":16128,"thread_start_us":1038,"threads_started":1}
I20260812 07:10:18.352916 3440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.316072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:41317 (local address 127.21.181.124:46377)
0812 07:10:18.317091 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.317097 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.326118 (+ 9021us) server_negotiation.cc:407] Connection header received
0812 07:10:18.326203 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.326209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.326304 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.326441 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.330735 (+ 4294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.332033 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.334475 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.335650 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.335750 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.343824 (+ 8074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.343832 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.343836 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.343879 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.347573 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.349597 (+ 2024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.349603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.349606 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.349770 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.350436 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.350442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.350447 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.350810 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.350951 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.352280 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.352535 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893,"thread_start_us":771,"threads_started":1}
I20260812 07:10:18.355379 3417 heartbeater.cc:347] Connected to a master server at 127.21.181.126:41905
I20260812 07:10:18.355376 3434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.315911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.21.181.66:41317)
0812 07:10:18.324707 (+ 8796us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:18.324724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:18.324841 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:18.329387 (+ 4546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:18.329391 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:18.329939 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:18.330509 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.330523 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.332750 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.332756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.334293 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.334309 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.336113 (+ 1804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.336120 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.336364 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.336863 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:18.336885 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:18.339597 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:18.347744 (+ 8147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.347751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.347755 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.349446 (+ 1691us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.349907 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.349913 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.349916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.350277 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.351019 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:18.351029 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:18.351217 (+ 188us) client_negotiation.cc:769] Sending connection context
0812 07:10:18.351369 (+ 152us) client_negotiation.cc:240] Negotiation successful
0812 07:10:18.351676 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8678,"thread_start_us":322,"threads_started":1}
I20260812 07:10:18.355549 3417 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:18.356006 3417 heartbeater.cc:511] Master 127.21.181.126:41905 requested a full tablet report, sending...
I20260812 07:10:18.356467 3426 heartbeater.cc:347] Connected to a master server at 127.21.181.124:46377
I20260812 07:10:18.356594 3426 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:18.356734 2346 ts_manager.cc:194] Registered new tserver with Master: 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441)
I20260812 07:10:18.357018 3426 heartbeater.cc:511] Master 127.21.181.124:46377 requested a full tablet report, sending...
I20260812 07:10:18.357957 1183 ts_manager.cc:194] Re-registered known tserver with Master: 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441)
I20260812 07:10:18.358096 2768 ts_manager.cc:194] Registered new tserver with Master: 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441)
I20260812 07:10:18.359406 1183 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:39773
I20260812 07:10:18.447355 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 1815
I20260812 07:10:18.470705 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:38483
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:41905
--webserver_port=36473
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:10:18.734854 3130 heartbeater.cc:503] Master 127.21.181.126:41905 was elected leader, sending a full tablet report...
W20260812 07:10:18.791503 3446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:18.791751 3446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:18.791849 3446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:18.798713 3446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:18.798802 3446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:18.798911 3446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:18.808205 3446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:38483
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=36473
--tserver_master_addrs=127.21.181.126:41905,127.21.181.125:37625,127.21.181.124:46377
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:18.809345 3446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:18.810559 3446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:18.822752 3453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:18.822759 3455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:18.823146 3458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:18.823800 3446 server_base.cc:1034] running on GCE node
I20260812 07:10:18.824386 3446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:18.825227 3446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:18.826419 3446 hybrid_clock.cc:648] HybridClock initialized: now 1786518618826372 us; error 36 us; skew 500 ppm
I20260812 07:10:18.826705 3446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:18.829679 3446 init.cc:377] Logged in from keytab as oryx/127.21.181.67@KRBTEST.COM (short username oryx)
I20260812 07:10:18.830971 3446 webserver.cc:460] Webserver started at http://127.21.181.67:36473/ using document root <none> and password file <none>
I20260812 07:10:18.831591 3446 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:18.831712 3446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:18.836737 3446 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:10:18.839191 3467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:18.840461 3446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:10:18.840605 3446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "c1d239dc082f41a4ae5eafce924400f3"
format_stamp: "Formatted at 2026-08-12 07:10:15 on dist-test-slave-lgr5"
server_key: "d9b04901691d1c58c4af0b6163647ede"
server_key_iv: "700d07f24e0dfc77d889953945fcb31b"
server_key_version: "encryptionkey@0"
I20260812 07:10:18.841535 3446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:18.854312 3446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:18.858462 3446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:18.858714 3446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:18.859643 3446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:18.860949 3446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:18.861038 3446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:18.861143 3446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:18.861215 3446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:18.889094 3446 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38483
I20260812 07:10:18.889153 3694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38483 every 8 connection(s)
I20260812 07:10:18.890516 3446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:18.893157 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 3446
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.124@KRBTEST.COM
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
Aug 12 07:10:18 dist-test-slave-lgr5 krb5kdc[1084](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518618, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.125@KRBTEST.COM
I20260812 07:10:18.940982 3711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.893985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.21.181.67:58475)
0812 07:10:18.909196 (+ 15211us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:18.909214 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:18.911623 (+ 2409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:18.914314 (+ 2691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:18.914318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:18.914707 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:18.915981 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.915991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.917451 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.917456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.919674 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.919694 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.920524 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.920532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.921821 (+ 1289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.922839 (+ 1018us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:18.922866 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:18.927854 (+ 4988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:18.932453 (+ 4599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.932463 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.932482 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.933081 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.935296 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.935302 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.935306 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.935612 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.937891 (+ 2279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:18.937901 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:18.938382 (+ 481us) client_negotiation.cc:769] Sending connection context
0812 07:10:18.938767 (+ 385us) client_negotiation.cc:240] Negotiation successful
0812 07:10:18.939359 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15059,"mutex_wait_us":114,"spinlock_wait_cycles":57728,"thread_start_us":15439,"threads_started":1}
I20260812 07:10:18.943463 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.910961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:47585)
0812 07:10:18.912806 (+ 1845us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:18.912824 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:18.912993 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:18.914955 (+ 1962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:18.914959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:18.915551 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:18.915926 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.915942 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.917477 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.917482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.920774 (+ 3292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.920789 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.922035 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.922042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.922236 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.924075 (+ 1839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:18.924124 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:18.927084 (+ 2960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:18.935880 (+ 8796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.935885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.935889 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.936424 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.936887 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.936892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.936896 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.937088 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.942398 (+ 5310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:18.942403 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:18.942578 (+ 175us) client_negotiation.cc:769] Sending connection context
0812 07:10:18.942722 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 07:10:18.943018 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1744,"mutex_wait_us":528,"thread_start_us":1603,"threads_started":1}
I20260812 07:10:18.944166 3709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.893985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.21.181.67:35655)
0812 07:10:18.908429 (+ 14444us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:18.908503 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:18.911987 (+ 3484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:18.913661 (+ 1674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:18.913672 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:18.914649 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:18.915574 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.915599 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.919077 (+ 3478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.919083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.920584 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:18.920604 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.921647 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.921655 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:18.921851 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.922704 (+ 853us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:18.922738 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:18.930111 (+ 7373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:18.939607 (+ 9496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.939611 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.939615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.940195 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.941372 (+ 1177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:18.941378 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:18.941382 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:18.941575 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:18.942846 (+ 1271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:18.942852 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:18.943017 (+ 165us) client_negotiation.cc:769] Sending connection context
0812 07:10:18.943160 (+ 143us) client_negotiation.cc:240] Negotiation successful
0812 07:10:18.943470 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14086,"mutex_wait_us":5,"thread_start_us":16366,"threads_started":1}
I20260812 07:10:18.945416 3719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.911092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:47585 (local address 127.21.181.126:41905)
0812 07:10:18.912412 (+ 1320us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.912418 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.912901 (+ 483us) server_negotiation.cc:407] Connection header received
0812 07:10:18.913142 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.913147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.913224 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.913302 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.916135 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.917335 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.920946 (+ 3611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.921669 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.921791 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.927274 (+ 5483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.927280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.927284 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.927324 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.934794 (+ 7470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.936579 (+ 1785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.936585 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.936589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.936754 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.941708 (+ 4954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.941715 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.941718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.942176 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.942329 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.944754 (+ 2425us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.945015 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1191,"thread_start_us":1066,"threads_started":1}
I20260812 07:10:18.945966 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.893429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:58475 (local address 127.21.181.125:37625)
0812 07:10:18.913831 (+ 20402us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.913838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.913858 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:10:18.913914 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.913919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.914008 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.914120 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.916161 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.917270 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.919806 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.920287 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.920400 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.928043 (+ 7643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.928050 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.928054 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.928151 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.932252 (+ 4101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.934946 (+ 2694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.934953 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.934958 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.935159 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.937219 (+ 2060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.937225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.937229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.937636 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.937794 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.943888 (+ 6094us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.944187 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20281,"thread_start_us":20174,"threads_started":1}
I20260812 07:10:18.947237 3697 heartbeater.cc:347] Connected to a master server at 127.21.181.126:41905
I20260812 07:10:18.947604 3697 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:18.948585 3697 heartbeater.cc:511] Master 127.21.181.126:41905 requested a full tablet report, sending...
I20260812 07:10:18.950498 1183 ts_manager.cc:194] Re-registered known tserver with Master: c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483)
I20260812 07:10:18.951665 1183 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:47585
I20260812 07:10:18.956154 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.893733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:35655 (local address 127.21.181.124:46377)
0812 07:10:18.913119 (+ 19386us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.913125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.913143 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:18.913197 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.913202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.913270 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.913376 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.916161 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.917270 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.920768 (+ 3498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.921427 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.922250 (+ 823us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.931673 (+ 9423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.931680 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.931685 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.931730 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.938424 (+ 6694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.940408 (+ 1984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.940414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.940418 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.940577 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.941763 (+ 1186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.941769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.941772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.942160 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.944809 (+ 2649us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.944943 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.945183 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19265,"thread_start_us":19119,"threads_started":1}
I20260812 07:10:18.957309 3698 heartbeater.cc:347] Connected to a master server at 127.21.181.124:46377
I20260812 07:10:18.957777 3698 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:18.958226 3698 heartbeater.cc:511] Master 127.21.181.124:46377 requested a full tablet report, sending...
I20260812 07:10:18.959923 2768 ts_manager.cc:194] Registered new tserver with Master: c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483)
I20260812 07:10:18.963126 3701 heartbeater.cc:347] Connected to a master server at 127.21.181.125:37625
I20260812 07:10:18.963251 3701 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:18.963600 3701 heartbeater.cc:511] Master 127.21.181.125:37625 requested a full tablet report, sending...
I20260812 07:10:18.964887 2346 ts_manager.cc:194] Registered new tserver with Master: c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483)
I20260812 07:10:18.968192 3721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.920796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.21.181.126:41905)
0812 07:10:18.923020 (+ 2224us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.923025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.926616 (+ 3591us) server_negotiation.cc:407] Connection header received
0812 07:10:18.926670 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.926675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.926742 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.926834 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.951873 (+ 25039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.953120 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.955035 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.955661 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.955775 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.961682 (+ 5907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.961688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.961693 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.961730 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.965216 (+ 3486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.966114 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.966119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.966123 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.966277 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.966681 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.966686 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.966690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.967023 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.967259 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.967568 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.967792 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2091,"thread_start_us":4638,"threads_started":1}
I20260812 07:10:18.970448 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.923452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37350 (local address 127.21.181.125:37625)
0812 07:10:18.946224 (+ 22772us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.946230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.948948 (+ 2718us) server_negotiation.cc:407] Connection header received
0812 07:10:18.949207 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.949213 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.949296 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.949362 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.952104 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.953120 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.957457 (+ 4337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.958170 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.958265 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.964233 (+ 5968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.964240 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.964245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.964281 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.967445 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.968302 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.968308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.968311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.968452 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.968841 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.968845 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.968847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.969165 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.969384 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.969886 (+ 502us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.970227 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22644,"thread_start_us":38958,"threads_started":1}
I20260812 07:10:18.976634 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.925521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54426 (local address 127.21.181.124:46377)
0812 07:10:18.960705 (+ 35184us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:18.960711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:18.960730 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:10:18.960779 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:18.960784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:18.960881 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:18.960976 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:18.963612 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.964821 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.966595 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:18.967025 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:18.967131 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:18.970920 (+ 3789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:18.970927 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:18.970930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:18.970968 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:18.974185 (+ 3217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.974741 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.974747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.974750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.974918 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:18.975226 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:18.975231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:18.975234 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:18.975566 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:18.975791 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:18.976062 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 07:10:18.976302 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35078,"thread_start_us":36089,"threads_started":1}
I20260812 07:10:18.972620 1183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:18.981078 1183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:19.024217 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.998514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38483 (local address 127.0.0.1:50684)
0812 07:10:19.012998 (+ 14484us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:19.013013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:19.013138 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:19.014496 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:19.014501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:19.014722 (+ 221us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:19.015104 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.015115 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.017351 (+ 2236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.017357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.019595 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.019611 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.022626 (+ 3015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.022634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.022874 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.022922 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 07:10:19.023040 (+ 118us) client_negotiation.cc:240] Negotiation successful
0812 07:10:19.023206 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14370,"spinlock_wait_cycles":417664,"thread_start_us":14720,"threads_started":1}
I20260812 07:10:19.030941 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.998869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.21.181.67:38483)
0812 07:10:19.012575 (+ 13706us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.012585 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.013421 (+ 836us) server_negotiation.cc:407] Connection header received
0812 07:10:19.013548 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.013558 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.013855 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.014057 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:19.015326 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.017151 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.020793 (+ 3642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.022368 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.022503 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.022722 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.030473 (+ 7751us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.030660 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13538,"thread_start_us":13345,"threads_started":1}
I20260812 07:10:19.031432 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.998350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.21.181.66:39441)
0812 07:10:19.022448 (+ 24098us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.022457 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.022487 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:10:19.022589 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.022596 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.022875 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.023099 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:19.023728 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.025777 (+ 2049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.028413 (+ 2636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.030372 (+ 1959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.030501 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.030696 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.031017 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.031175 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23921,"thread_start_us":24910,"threads_started":1}
I20260812 07:10:19.031816 3756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.998070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39441 (local address 127.0.0.1:45024)
0812 07:10:19.011982 (+ 13912us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:19.012003 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:19.014607 (+ 2604us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:19.023083 (+ 8476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:19.023088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:19.023113 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:19.023549 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.023561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.025963 (+ 2402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.025968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.028243 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.028259 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.030593 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.030601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.030804 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.030845 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:10:19.030952 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:10:19.031278 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13763,"thread_start_us":32131,"threads_started":1}
I20260812 07:10:19.034263 3759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.999306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:44169 (local address 127.0.0.1:42322)
0812 07:10:19.024870 (+ 25564us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:19.024892 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:19.025016 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:19.026055 (+ 1039us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:19.026061 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:19.026094 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:19.026415 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.026428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.028937 (+ 2509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.028945 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.031858 (+ 2913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.031876 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.033448 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.033455 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.033661 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.033696 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:10:19.033793 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:10:19.033941 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25449,"thread_start_us":13437,"threads_started":1}
I20260812 07:10:19.034562 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:18.999628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42322 (local address 127.21.181.65:44169)
0812 07:10:19.020178 (+ 20550us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.020187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.025181 (+ 4994us) server_negotiation.cc:407] Connection header received
0812 07:10:19.025320 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.025329 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.025583 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.025785 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:19.026636 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.028369 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.032180 (+ 3811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.033247 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.033430 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.033655 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.033851 (+ 196us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.034021 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20340,"thread_start_us":20216,"threads_started":1}
I20260812 07:10:19.036458 3293 tablet_service.cc:1467] Processing CreateTablet for tablet 2dca1fadf59349109587234fa41846f0 (DEFAULT_TABLE table=test-table [id=93adf6454e924d35907131c1330e83f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:19.038076 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dca1fadf59349109587234fa41846f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:19.038787 2980 tablet_service.cc:1467] Processing CreateTablet for tablet 2dca1fadf59349109587234fa41846f0 (DEFAULT_TABLE table=test-table [id=93adf6454e924d35907131c1330e83f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:19.040678 2980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dca1fadf59349109587234fa41846f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:19.041602 3569 tablet_service.cc:1467] Processing CreateTablet for tablet 2dca1fadf59349109587234fa41846f0 (DEFAULT_TABLE table=test-table [id=93adf6454e924d35907131c1330e83f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:19.042908 3569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dca1fadf59349109587234fa41846f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:19.059392 3768 tablet_bootstrap.cc:492] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Bootstrap starting.
I20260812 07:10:19.060913 3766 tablet_bootstrap.cc:492] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Bootstrap starting.
I20260812 07:10:19.062703 3770 tablet_bootstrap.cc:492] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Bootstrap starting.
I20260812 07:10:19.063093 3768 tablet_bootstrap.cc:654] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:19.064150 3768 log.cc:824] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:19.065233 3766 tablet_bootstrap.cc:654] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:19.066713 3766 log.cc:824] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:19.066833 3770 tablet_bootstrap.cc:654] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:19.067421 3768 tablet_bootstrap.cc:492] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: No bootstrap required, opened a new log
I20260812 07:10:19.067693 3768 ts_tablet_manager.cc:1397] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 07:10:19.068150 3770 log.cc:824] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:19.069662 3766 tablet_bootstrap.cc:492] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: No bootstrap required, opened a new log
I20260812 07:10:19.069968 3766 ts_tablet_manager.cc:1397] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 07:10:19.071213 3770 tablet_bootstrap.cc:492] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: No bootstrap required, opened a new log
I20260812 07:10:19.071517 3770 ts_tablet_manager.cc:1397] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260812 07:10:19.074970 3768 raft_consensus.cc:348] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.075325 3768 raft_consensus.cc:374] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:19.075430 3768 raft_consensus.cc:729] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c883c5578a264138870f8efc4d129bd2, State: Initialized, Role: FOLLOWER
I20260812 07:10:19.075989 3768 consensus_queue.cc:260] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.080845 3766 raft_consensus.cc:348] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.081317 3766 raft_consensus.cc:374] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:19.081486 3766 raft_consensus.cc:729] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1050870805f441d287bd01e27acedd9a, State: Initialized, Role: FOLLOWER
I20260812 07:10:19.082113 3766 consensus_queue.cc:260] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.082172 3770 raft_consensus.cc:348] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.082619 3770 raft_consensus.cc:374] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:19.082736 3770 raft_consensus.cc:729] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d239dc082f41a4ae5eafce924400f3, State: Initialized, Role: FOLLOWER
I20260812 07:10:19.083303 3417 heartbeater.cc:503] Master 127.21.181.126:41905 was elected leader, sending a full tablet report...
I20260812 07:10:19.083426 3770 consensus_queue.cc:260] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.083871 3768 ts_tablet_manager.cc:1428] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Time spent starting tablet: real 0.016s user 0.012s sys 0.004s
I20260812 07:10:19.084338 3766 ts_tablet_manager.cc:1428] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Time spent starting tablet: real 0.014s user 0.009s sys 0.007s
I20260812 07:10:19.086846 3697 heartbeater.cc:503] Master 127.21.181.126:41905 was elected leader, sending a full tablet report...
I20260812 07:10:19.091935 3770 ts_tablet_manager.cc:1428] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Time spent starting tablet: real 0.020s user 0.004s sys 0.011s
I20260812 07:10:19.102521 3784 raft_consensus.cc:482] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:19.102823 3784 raft_consensus.cc:504] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.104465 3784 leader_election.cc:288] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483), c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:19.112283 3760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.104882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:48753 (local address 127.21.181.65:44169)
0812 07:10:19.105080 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.105087 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.105815 (+ 728us) server_negotiation.cc:407] Connection header received
0812 07:10:19.105988 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.105994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.106067 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.106142 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:19.106677 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.107679 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.109993 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.111326 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.111469 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.111539 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.111852 (+ 313us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.111982 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:10:19.112283 3757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.104684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:52719 (local address 127.21.181.67:38483)
0812 07:10:19.104864 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.104870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.105520 (+ 650us) server_negotiation.cc:407] Connection header received
0812 07:10:19.105748 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.105752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.105814 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.105890 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:19.106366 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.107429 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.109961 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.111235 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.111430 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.111517 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.111853 (+ 336us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.111982 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:10:19.112439 3794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.105079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:44169 (local address 127.21.181.66:48753)
0812 07:10:19.105724 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:19.105736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:19.105838 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:19.106189 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:19.106197 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:19.106219 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:19.106511 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.106519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.107794 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.107799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.109847 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.109858 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.111519 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.111525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.111685 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.111740 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 07:10:19.111817 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:10:19.111998 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"mutex_wait_us":76,"thread_start_us":373,"threads_started":1}
I20260812 07:10:19.112759 3793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.104612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38483 (local address 127.21.181.66:52719)
0812 07:10:19.105394 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:19.105414 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:19.105586 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:19.105910 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:19.105914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:19.105938 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:19.106216 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.106227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.107565 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.107569 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.109780 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:19.109797 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.111447 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.111454 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:19.111669 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.111724 (+ 55us) client_negotiation.cc:769] Sending connection context
0812 07:10:19.111817 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:10:19.111988 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":31,"thread_start_us":408,"threads_started":1}
I20260812 07:10:19.113400 3013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dca1fadf59349109587234fa41846f0" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c883c5578a264138870f8efc4d129bd2" is_pre_election: true
I20260812 07:10:19.113526 3601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dca1fadf59349109587234fa41846f0" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d239dc082f41a4ae5eafce924400f3" is_pre_election: true
I20260812 07:10:19.113894 3013 raft_consensus.cc:2454] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 0.
I20260812 07:10:19.114092 3601 raft_consensus.cc:2454] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 0.
I20260812 07:10:19.114584 3191 leader_election.cc:302] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1050870805f441d287bd01e27acedd9a, c883c5578a264138870f8efc4d129bd2; no voters:
I20260812 07:10:19.115195 3784 raft_consensus.cc:2781] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:19.115334 3784 raft_consensus.cc:482] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:19.115435 3784 raft_consensus.cc:3037] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:19.117882 3784 raft_consensus.cc:504] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.118522 3784 leader_election.cc:288] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 election: Requested vote from peers c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483), c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:19.118781 3601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dca1fadf59349109587234fa41846f0" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d239dc082f41a4ae5eafce924400f3"
I20260812 07:10:19.118942 3601 raft_consensus.cc:3037] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:19.119000 3013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dca1fadf59349109587234fa41846f0" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c883c5578a264138870f8efc4d129bd2"
I20260812 07:10:19.119200 3013 raft_consensus.cc:3037] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:19.121192 3601 raft_consensus.cc:2454] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 1.
I20260812 07:10:19.121563 3013 raft_consensus.cc:2454] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 1.
I20260812 07:10:19.121563 3188 leader_election.cc:302] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1050870805f441d287bd01e27acedd9a, c1d239dc082f41a4ae5eafce924400f3; no voters:
I20260812 07:10:19.121869 3784 raft_consensus.cc:2781] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:19.122285 3784 raft_consensus.cc:686] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 1 LEADER]: Becoming Leader. State: Replica: 1050870805f441d287bd01e27acedd9a, State: Running, Role: LEADER
I20260812 07:10:19.122768 3784 consensus_queue.cc:237] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:19.127283 1183 catalog_manager.cc:5495] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1050870805f441d287bd01e27acedd9a (127.21.181.66). New cstate: current_term: 1 leader_uuid: "1050870805f441d287bd01e27acedd9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } health_report { overall_health: UNKNOWN } } }
W20260812 07:10:19.143340 3703 tablet.cc:2369] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:19.150486 3754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.145460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.21.181.66:39441)
0812 07:10:19.145656 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.145662 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.145680 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:19.145771 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.145776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.145855 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.145988 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:19.146392 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.147090 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.148418 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.148811 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.148926 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.149405 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:19.149921 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:19.150061 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.150162 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.150241 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:10:19.158192 3013 raft_consensus.cc:1261] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Refusing update from remote peer 1050870805f441d287bd01e27acedd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:19.158192 3601 raft_consensus.cc:1261] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Refusing update from remote peer 1050870805f441d287bd01e27acedd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:19.159178 3795 consensus_queue.cc:1035] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:19.159502 3784 consensus_queue.cc:1035] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:19.171989 3808 mvcc.cc:205] Tried to move back new op lower bound from 7317580264062738432 to 7317580263930454016. Current Snapshot: MvccSnapshot[applied={T|T < 7317580264062738432}]
I20260812 07:10:19.175616 3807 mvcc.cc:205] Tried to move back new op lower bound from 7317580264062738432 to 7317580263930454016. Current Snapshot: MvccSnapshot[applied={T|T < 7317580264062738432}]
I20260812 07:10:19.197353 1183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:19.197573 1183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:19.205492 1183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad16eee3ad2d49539a1039a65084d1ac: Sending DeleteTablet for 3 replicas of tablet 2dca1fadf59349109587234fa41846f0
I20260812 07:10:19.206535 3293 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dca1fadf59349109587234fa41846f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:19 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:45024
I20260812 07:10:19.206645 3569 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dca1fadf59349109587234fa41846f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:19 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:50684
I20260812 07:10:19.207325 2980 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dca1fadf59349109587234fa41846f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:19 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:42322
I20260812 07:10:19.208241 3826 tablet_replica.cc:330] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: stopping tablet replica
I20260812 07:10:19.208663 3826 raft_consensus.cc:2229] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:19.209175 3826 raft_consensus.cc:2258] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:19.209863 3818 tablet_replica.cc:330] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: stopping tablet replica
I20260812 07:10:19.210217 3818 raft_consensus.cc:2229] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:19.210821 3818 raft_consensus.cc:2258] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:19.211206 3826 ts_tablet_manager.cc:1905] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:19.213760 3818 ts_tablet_manager.cc:1905] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:19.218240 3826 ts_tablet_manager.cc:1918] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:19.218389 3826 log.cc:1196] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/2dca1fadf59349109587234fa41846f0
I20260812 07:10:19.218953 3826 ts_tablet_manager.cc:1939] T 2dca1fadf59349109587234fa41846f0 P c883c5578a264138870f8efc4d129bd2: Deleting consensus metadata
I20260812 07:10:19.219245 3822 tablet_replica.cc:330] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: stopping tablet replica
I20260812 07:10:19.220772 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 1116
I20260812 07:10:19.220752 1154 catalog_manager.cc:4841] TS c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169): tablet 2dca1fadf59349109587234fa41846f0 (table test-table [id=93adf6454e924d35907131c1330e83f2]) successfully deleted
I20260812 07:10:19.221794 1241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:19.224385 3822 raft_consensus.cc:2229] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:19.224774 3822 raft_consensus.cc:2258] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:19.226526 3822 ts_tablet_manager.cc:1905] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:19.232956 3818 ts_tablet_manager.cc:1918] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:19.233217 3818 log.cc:1196] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/2dca1fadf59349109587234fa41846f0
I20260812 07:10:19.233671 3822 ts_tablet_manager.cc:1918] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:19.233831 3822 log.cc:1196] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/2dca1fadf59349109587234fa41846f0
I20260812 07:10:19.234035 3818 ts_tablet_manager.cc:1939] T 2dca1fadf59349109587234fa41846f0 P 1050870805f441d287bd01e27acedd9a: Deleting consensus metadata
I20260812 07:10:19.234341 3822 ts_tablet_manager.cc:1939] T 2dca1fadf59349109587234fa41846f0 P c1d239dc082f41a4ae5eafce924400f3: Deleting consensus metadata
I20260812 07:10:19.242728 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.210178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37362 (local address 127.21.181.125:37625)
0812 07:10:19.210337 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.210343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.219386 (+ 9043us) server_negotiation.cc:407] Connection header received
0812 07:10:19.219453 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.219459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.219539 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.219641 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:19.223671 (+ 4030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.224862 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.226955 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.227605 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.227708 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.235469 (+ 7761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:19.235488 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:19.235493 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:19.235536 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:19.239405 (+ 3869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.240263 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.240270 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.240274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.240442 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.240891 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.240897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.240900 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.241346 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:19.241562 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.242158 (+ 596us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.242396 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:10:19.261798 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.219163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.21.181.124:46377)
0812 07:10:19.219340 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.219346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.244416 (+ 25070us) server_negotiation.cc:407] Connection header received
0812 07:10:19.244495 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.244500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.244582 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.244669 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:19.246413 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.247601 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.249527 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.250171 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.250306 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.253583 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:19.253591 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:19.253597 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:19.253641 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:19.257165 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.258002 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.258009 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.258012 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.258533 (+ 521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.259394 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.259399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.259402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.259782 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:19.260028 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.261168 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.261402 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:10:19.361326 1153 catalog_manager.cc:4841] TS c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483): tablet 2dca1fadf59349109587234fa41846f0 (table test-table [id=93adf6454e924d35907131c1330e83f2]) successfully deleted
I20260812 07:10:19.362028 1159 catalog_manager.cc:4841] TS 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441): tablet 2dca1fadf59349109587234fa41846f0 (table test-table [id=93adf6454e924d35907131c1330e83f2]) successfully deleted
I20260812 07:10:19.362633 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 1116
I20260812 07:10:19.418943 3706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.404063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54434 (local address 127.21.181.124:46377)
0812 07:10:19.404282 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.404286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.404652 (+ 366us) server_negotiation.cc:407] Connection header received
0812 07:10:19.404862 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.404866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.404929 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.405021 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:19.407193 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.408336 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.409648 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.410237 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.410351 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.413316 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:19.413322 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:19.413332 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:19.413375 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:19.416155 (+ 2780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.416788 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.416794 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.416798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.416923 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.417256 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.417261 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.417264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.417562 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:19.417809 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.418396 (+ 587us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.418617 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260812 07:10:19.420722 3704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:19.402629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37364 (local address 127.21.181.125:37625)
0812 07:10:19.402863 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:19.402869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:19.403578 (+ 709us) server_negotiation.cc:407] Connection header received
0812 07:10:19.403915 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:19.403921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:19.404005 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:19.404117 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:19.406532 (+ 2415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.407504 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.408900 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:19.409574 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:19.409705 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:19.414089 (+ 4384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:19.414097 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:19.414102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:19.414141 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:19.417508 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.418250 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.418255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.418259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.418418 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:19.418859 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:19.418865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:19.418868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:19.419467 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:19.419688 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:19.420300 (+ 612us) server_negotiation.cc:299] Negotiation successful
0812 07:10:19.420464 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:10:20.181489 3840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.180678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:37575)
0812 07:10:20.181177 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.181307 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":235,"threads_started":1}
W20260812 07:10:20.181897 3697 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:41905 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.185513 3842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.184860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:56113)
0812 07:10:20.185249 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.185357 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":199,"threads_started":1}
W20260812 07:10:20.185860 3130 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:41905 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.193310 3845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.192550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:47097)
0812 07:10:20.192957 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.193156 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":213,"threads_started":1}
W20260812 07:10:20.193621 3417 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:41905 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.704773 3851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad16eee3ad2d49539a1039a65084d1ac)
I20260812 07:10:20.705019 3851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } }
I20260812 07:10:20.705791 3852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Starting pre-election (detected failure of leader ad16eee3ad2d49539a1039a65084d1ac)
I20260812 07:10:20.706033 3852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } }
I20260812 07:10:20.706439 3851 leader_election.cc:288] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905), 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125:37625)
I20260812 07:10:20.707487 3852 leader_election.cc:288] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905), 176509da617f4f9f95ed259d9f9766a1 (127.21.181.124:46377)
I20260812 07:10:20.708571 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.706588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45492)
0812 07:10:20.708048 (+ 1460us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.708284 (+ 236us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1313,"thread_start_us":1177,"threads_started":1}
W20260812 07:10:20.708855 2740 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.709712 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.708707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45508)
0812 07:10:20.709267 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.709440 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":246,"threads_started":1}
I20260812 07:10:20.711928 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.711567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45512)
0812 07:10:20.711717 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.711797 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:10:20.712144 2740 leader_election.cc:334] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905): Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
W20260812 07:10:20.713321 2321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.714888 3867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.714454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45528)
0812 07:10:20.714619 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.714744 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:10:20.715106 2321 leader_election.cc:334] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905): Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.717036 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.706727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:37625 (local address 127.0.0.1:34016)
0812 07:10:20.708364 (+ 1637us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.708381 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:20.708521 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:20.710227 (+ 1706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:20.710233 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:20.710265 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:20.710669 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.710682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.712962 (+ 2280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.712968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.715105 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.715119 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.716441 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.716445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.716572 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.716597 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:10:20.716670 (+ 73us) client_negotiation.cc:240] Negotiation successful
0812 07:10:20.716755 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1520,"mutex_wait_us":1,"thread_start_us":1332,"threads_started":1}
I20260812 07:10:20.717198 3855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.706722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34016 (local address 127.21.181.125:37625)
0812 07:10:20.708705 (+ 1983us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:20.708711 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:20.708729 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:20.708791 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:20.708795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:20.708891 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:20.708987 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:20.711184 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.712813 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.715280 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.716291 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.716438 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.716516 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:20.716720 (+ 204us) server_negotiation.cc:299] Negotiation successful
0812 07:10:20.716848 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1861,"thread_start_us":1693,"threads_started":1}
I20260812 07:10:20.717729 2362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "176509da617f4f9f95ed259d9f9766a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" is_pre_election: true
I20260812 07:10:20.717994 2362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 176509da617f4f9f95ed259d9f9766a1 in term 1.
I20260812 07:10:20.718487 2745 leader_election.cc:302] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 176509da617f4f9f95ed259d9f9766a1, 4f81a580229a4d7e8f92cbe0bc2d7bab; no voters: ad16eee3ad2d49539a1039a65084d1ac
I20260812 07:10:20.718883 3851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 07:10:20.719046 3851 raft_consensus.cc:482] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader ad16eee3ad2d49539a1039a65084d1ac)
I20260812 07:10:20.719173 3851 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:10:20.719769 3869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.709479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.21.181.124:46377)
0812 07:10:20.713575 (+ 4096us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:20.713581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:20.713607 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:10:20.713666 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:20.713670 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:20.713738 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:20.714448 (+ 710us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:20.714735 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.715724 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.717820 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.718881 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.719059 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.719116 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:20.719431 (+ 315us) server_negotiation.cc:299] Negotiation successful
0812 07:10:20.719524 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3947,"thread_start_us":3819,"threads_started":1}
I20260812 07:10:20.719774 3864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.709305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:46377 (local address 127.0.0.1:60506)
0812 07:10:20.709904 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.709921 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:20.710054 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:20.713906 (+ 3852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:20.713912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:20.713945 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:20.714311 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.714323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.715847 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.715853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.717674 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.717687 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.719095 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.719102 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.719278 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.719311 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:10:20.719409 (+ 98us) client_negotiation.cc:240] Negotiation successful
0812 07:10:20.719535 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":3732,"threads_started":1}
I20260812 07:10:20.720436 2792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "176509da617f4f9f95ed259d9f9766a1" is_pre_election: true
I20260812 07:10:20.721800 3851 raft_consensus.cc:504] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } }
I20260812 07:10:20.722198 2792 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4f81a580229a4d7e8f92cbe0bc2d7bab in current term 2: Already voted for candidate 176509da617f4f9f95ed259d9f9766a1 in this term.
I20260812 07:10:20.722434 3851 leader_election.cc:288] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [CANDIDATE]: Term 2 election: Requested vote from peers ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905), 4f81a580229a4d7e8f92cbe0bc2d7bab (127.21.181.125:37625)
I20260812 07:10:20.722851 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.722502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45534)
0812 07:10:20.722661 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.722740 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:10:20.722982 2362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "176509da617f4f9f95ed259d9f9766a1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab"
I20260812 07:10:20.723110 2321 leader_election.cc:302] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4f81a580229a4d7e8f92cbe0bc2d7bab; no voters: 176509da617f4f9f95ed259d9f9766a1, ad16eee3ad2d49539a1039a65084d1ac
I20260812 07:10:20.723196 2362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:10:20.723774 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.723430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45538)
0812 07:10:20.723565 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.723644 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260812 07:10:20.723939 2740 leader_election.cc:334] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905): Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
I20260812 07:10:20.725481 2362 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 176509da617f4f9f95ed259d9f9766a1 in term 2.
I20260812 07:10:20.725652 3852 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 07:10:20.725854 2745 leader_election.cc:302] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 176509da617f4f9f95ed259d9f9766a1, 4f81a580229a4d7e8f92cbe0bc2d7bab; no voters: ad16eee3ad2d49539a1039a65084d1ac
I20260812 07:10:20.726107 3851 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:10:20.726500 3851 raft_consensus.cc:686] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [term 2 LEADER]: Becoming Leader. State: Replica: 176509da617f4f9f95ed259d9f9766a1, State: Running, Role: LEADER
I20260812 07:10:20.726830 3851 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } }
I20260812 07:10:20.728288 3872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176509da617f4f9f95ed259d9f9766a1. Latest consensus state: current_term: 2 leader_uuid: "176509da617f4f9f95ed259d9f9766a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:20.728644 3872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:20.729239 3874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:20.732890 3874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:20.733830 3874 catalog_manager.cc:1259] Loaded cluster ID: 82424b418be74224ace6b4218c4e4b12
I20260812 07:10:20.733908 3874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:20.735036 3874 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:20.735821 3874 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Loaded TSK: 0
I20260812 07:10:20.736629 3874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:10:20.788558 2765 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:20.795529 2765 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:20.799531 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.799081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45564)
0812 07:10:20.799242 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.799352 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:10:20.799711 2362 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [term 2 FOLLOWER]: Refusing update from remote peer 176509da617f4f9f95ed259d9f9766a1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260812 07:10:20.800616 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.800204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45580)
0812 07:10:20.800381 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.800473 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:10:20.800570 3872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260812 07:10:20.800817 2740 consensus_peers.cc:487] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 -> Peer ad16eee3ad2d49539a1039a65084d1ac (127.21.181.126:41905): Couldn't send request to peer ad16eee3ad2d49539a1039a65084d1ac. Status: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 07:10:20.803158 3852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176509da617f4f9f95ed259d9f9766a1. Latest consensus state: current_term: 2 leader_uuid: "176509da617f4f9f95ed259d9f9766a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:20.803416 3852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:20.804817 3872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "176509da617f4f9f95ed259d9f9766a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:20.805114 3872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:20.805811 3852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "176509da617f4f9f95ed259d9f9766a1" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "ad16eee3ad2d49539a1039a65084d1ac" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41905 } } peers { permanent_uuid: "4f81a580229a4d7e8f92cbe0bc2d7bab" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 37625 } attrs { promote: false } } peers { permanent_uuid: "176509da617f4f9f95ed259d9f9766a1" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 46377 } attrs { promote: false } } }
I20260812 07:10:20.806085 3852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f81a580229a4d7e8f92cbe0bc2d7bab [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:10:20.806322 3878 mvcc.cc:205] Tried to move back new op lower bound from 7317580270788734976 to 7317580270500093952. Current Snapshot: MvccSnapshot[applied={T|T < 7317580270788734976 or (T in {7317580270788734976})}]
I20260812 07:10:20.819540 3859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.813093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38483 (local address 127.0.0.1:58882)
0812 07:10:20.813555 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.813574 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:20.813681 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:20.814110 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:20.814115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:20.814138 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:20.814312 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.814323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.815731 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.815737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.817442 (+ 1705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.817457 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.818806 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.818812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.819000 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.819025 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:10:20.819113 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:10:20.819238 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358}
I20260812 07:10:20.819552 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.813273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58882 (local address 127.21.181.67:38483)
0812 07:10:20.813760 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:20.813766 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:20.813786 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:10:20.813863 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:20.813868 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:20.813955 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:20.814459 (+ 504us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:20.814661 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.815609 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.817673 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.818663 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.818779 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.818886 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:20.819155 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:10:20.819289 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":1202,"threads_started":1}
I20260812 07:10:20.820385 3569 tablet_service.cc:1467] Processing CreateTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 (DEFAULT_TABLE table=test-table [id=e9014c0c46ad404bb5193b80187c99f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:20.820583 3853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.813410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39441 (local address 127.0.0.1:60246)
0812 07:10:20.813911 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.813928 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:20.814054 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:20.814442 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:20.814447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:20.814472 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:20.814816 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.814830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.816323 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.816329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.818283 (+ 1954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.818296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.819835 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.819841 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.820026 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.820064 (+ 38us) client_negotiation.cc:769] Sending connection context
0812 07:10:20.820173 (+ 109us) client_negotiation.cc:240] Negotiation successful
0812 07:10:20.820330 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":1}
I20260812 07:10:20.820650 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.813668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60246 (local address 127.21.181.66:39441)
0812 07:10:20.814114 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:20.814121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:20.814142 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:10:20.814213 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:20.814218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:20.814293 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:20.814964 (+ 671us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:20.815150 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.816150 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.818480 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.819641 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.819758 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.819845 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:20.820204 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 07:10:20.820347 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":456,"threads_started":1}
I20260812 07:10:20.820883 3569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb48cecd8464fedbff9f68eeb3ddc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:20.821336 3293 tablet_service.cc:1467] Processing CreateTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 (DEFAULT_TABLE table=test-table [id=e9014c0c46ad404bb5193b80187c99f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:20.821768 3293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb48cecd8464fedbff9f68eeb3ddc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:20.824571 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.813962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:44169 (local address 127.0.0.1:59490)
0812 07:10:20.816865 (+ 2903us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.816882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:20.817014 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:20.817485 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:20.817489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:20.817518 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:20.817926 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.817936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.819306 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.819311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.822229 (+ 2918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:20.822244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.823831 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.823840 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:20.824005 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.824039 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:10:20.824151 (+ 112us) client_negotiation.cc:240] Negotiation successful
0812 07:10:20.824262 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2786,"thread_start_us":2623,"threads_started":1}
I20260812 07:10:20.825248 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.814347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59490 (local address 127.21.181.65:44169)
0812 07:10:20.816431 (+ 2084us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:20.816437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:20.816971 (+ 534us) server_negotiation.cc:407] Connection header received
0812 07:10:20.817130 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:20.817136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:20.817213 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:20.817312 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:20.818114 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.819153 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.822409 (+ 3256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.823583 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.823705 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.823808 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:20.824813 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0812 07:10:20.824955 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1963,"thread_start_us":1875,"threads_started":1}
I20260812 07:10:20.825856 2980 tablet_service.cc:1467] Processing CreateTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 (DEFAULT_TABLE table=test-table [id=e9014c0c46ad404bb5193b80187c99f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:20.826280 2980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfb48cecd8464fedbff9f68eeb3ddc07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:20.827351 3900 tablet_bootstrap.cc:492] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Bootstrap starting.
I20260812 07:10:20.828162 3896 tablet_bootstrap.cc:492] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Bootstrap starting.
I20260812 07:10:20.829057 3900 tablet_bootstrap.cc:654] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:20.829806 3896 tablet_bootstrap.cc:654] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:20.831521 3900 tablet_bootstrap.cc:492] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: No bootstrap required, opened a new log
I20260812 07:10:20.831655 3900 ts_tablet_manager.cc:1397] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:10:20.832327 3900 raft_consensus.cc:348] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.832510 3900 raft_consensus.cc:374] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:20.832598 3900 raft_consensus.cc:729] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1050870805f441d287bd01e27acedd9a, State: Initialized, Role: FOLLOWER
I20260812 07:10:20.832815 3900 consensus_queue.cc:260] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.833561 3900 ts_tablet_manager.cc:1428] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:10:20.834640 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.833784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:36411)
0812 07:10:20.834232 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.834408 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"threads_started":1}
I20260812 07:10:20.835839 3903 tablet_bootstrap.cc:492] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Bootstrap starting.
I20260812 07:10:20.837549 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.836627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:47609)
0812 07:10:20.837336 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.837418 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":612}
I20260812 07:10:20.837610 3903 tablet_bootstrap.cc:654] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:20.838630 3426 heartbeater.cc:503] Master 127.21.181.124:46377 was elected leader, sending a full tablet report...
I20260812 07:10:20.840518 3896 tablet_bootstrap.cc:492] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: No bootstrap required, opened a new log
I20260812 07:10:20.840660 3896 ts_tablet_manager.cc:1397] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260812 07:10:20.840878 3903 tablet_bootstrap.cc:492] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: No bootstrap required, opened a new log
I20260812 07:10:20.841004 3903 ts_tablet_manager.cc:1397] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:10:20.841354 3896 raft_consensus.cc:348] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.841544 3896 raft_consensus.cc:374] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:20.841640 3896 raft_consensus.cc:729] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d239dc082f41a4ae5eafce924400f3, State: Initialized, Role: FOLLOWER
I20260812 07:10:20.841642 3903 raft_consensus.cc:348] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.841864 3903 raft_consensus.cc:374] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:20.841955 3903 raft_consensus.cc:729] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c883c5578a264138870f8efc4d129bd2, State: Initialized, Role: FOLLOWER
I20260812 07:10:20.841871 3896 consensus_queue.cc:260] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.842190 3903 consensus_queue.cc:260] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.842669 3896 ts_tablet_manager.cc:1428] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260812 07:10:20.842931 3903 ts_tablet_manager.cc:1428] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:10:20.848999 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.842900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:60893)
0812 07:10:20.848612 (+ 5712us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.848753 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5557,"thread_start_us":5431,"threads_started":1}
I20260812 07:10:20.850291 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.849949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:42467)
0812 07:10:20.850084 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.850163 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260812 07:10:20.850262 3698 heartbeater.cc:503] Master 127.21.181.124:46377 was elected leader, sending a full tablet report...
I20260812 07:10:20.854568 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.843403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:49315)
0812 07:10:20.854269 (+ 10866us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.854389 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10767,"threads_started":1}
I20260812 07:10:20.855645 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.855319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:53617)
0812 07:10:20.855464 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.855539 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:10:20.856190 3131 heartbeater.cc:503] Master 127.21.181.124:46377 was elected leader, sending a full tablet report...
I20260812 07:10:20.934228 3909 raft_consensus.cc:482] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:20.934438 3909 raft_consensus.cc:504] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.935227 3909 leader_election.cc:288] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483), c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:20.935763 3013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb48cecd8464fedbff9f68eeb3ddc07" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c883c5578a264138870f8efc4d129bd2" is_pre_election: true
I20260812 07:10:20.935750 3598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb48cecd8464fedbff9f68eeb3ddc07" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d239dc082f41a4ae5eafce924400f3" is_pre_election: true
I20260812 07:10:20.935998 3013 raft_consensus.cc:2454] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 0.
I20260812 07:10:20.936010 3598 raft_consensus.cc:2454] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 0.
I20260812 07:10:20.936473 3191 leader_election.cc:302] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1050870805f441d287bd01e27acedd9a, c883c5578a264138870f8efc4d129bd2; no voters:
I20260812 07:10:20.936728 3909 raft_consensus.cc:2781] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:20.936839 3909 raft_consensus.cc:482] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:20.936926 3909 raft_consensus.cc:3037] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:20.939013 3909 raft_consensus.cc:504] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.939513 3909 leader_election.cc:288] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 election: Requested vote from peers c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483), c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169)
I20260812 07:10:20.939982 3598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb48cecd8464fedbff9f68eeb3ddc07" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c1d239dc082f41a4ae5eafce924400f3"
I20260812 07:10:20.939992 3013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfb48cecd8464fedbff9f68eeb3ddc07" candidate_uuid: "1050870805f441d287bd01e27acedd9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c883c5578a264138870f8efc4d129bd2"
I20260812 07:10:20.940181 3598 raft_consensus.cc:3037] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:20.940213 3013 raft_consensus.cc:3037] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:20.942171 3013 raft_consensus.cc:2454] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 1.
I20260812 07:10:20.942180 3598 raft_consensus.cc:2454] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1050870805f441d287bd01e27acedd9a in term 1.
I20260812 07:10:20.942569 3191 leader_election.cc:302] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1050870805f441d287bd01e27acedd9a, c1d239dc082f41a4ae5eafce924400f3, c883c5578a264138870f8efc4d129bd2; no voters:
I20260812 07:10:20.942855 3909 raft_consensus.cc:2781] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:20.943274 3909 raft_consensus.cc:686] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 1 LEADER]: Becoming Leader. State: Replica: 1050870805f441d287bd01e27acedd9a, State: Running, Role: LEADER
I20260812 07:10:20.943568 3909 consensus_queue.cc:237] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } }
I20260812 07:10:20.944331 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.943832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:47347)
0812 07:10:20.943985 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.944112 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 07:10:20.946460 2765 catalog_manager.cc:5495] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1050870805f441d287bd01e27acedd9a (127.21.181.66). New cstate: current_term: 1 leader_uuid: "1050870805f441d287bd01e27acedd9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1050870805f441d287bd01e27acedd9a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 } health_report { overall_health: UNKNOWN } } }
W20260812 07:10:20.953332 3140 tablet.cc:2369] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:20.974398 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.969957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60262 (local address 127.21.181.66:39441)
0812 07:10:20.970146 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:20.970152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:20.970170 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:20.970332 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:20.970335 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:20.970396 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:20.970470 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:20.970821 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.971599 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.972867 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:20.973285 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:20.973395 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:20.973770 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:20.973874 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:20.973949 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:20.974092 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 07:10:20.974167 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:10:20.977082 3013 raft_consensus.cc:1261] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Refusing update from remote peer 1050870805f441d287bd01e27acedd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:20.977162 3598 raft_consensus.cc:1261] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Refusing update from remote peer 1050870805f441d287bd01e27acedd9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:20.977703 3928 consensus_queue.cc:1035] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c883c5578a264138870f8efc4d129bd2" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 44169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:20.978009 3909 consensus_queue.cc:1035] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c1d239dc082f41a4ae5eafce924400f3" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:20.980526 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.979965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:38459)
0812 07:10:20.980215 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.980312 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260812 07:10:20.980700 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.979973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:58573)
0812 07:10:20.980310 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.980452 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
I20260812 07:10:20.986565 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.986138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:60095)
0812 07:10:20.986329 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.986418 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:10:20.988718 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.988330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.65:59791)
0812 07:10:20.988482 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.988562 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 07:10:20.991840 2765 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:20.992120 2765 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:20.993973 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.992698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.67:41515)
0812 07:10:20.993740 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.993831 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":936}
I20260812 07:10:20.994872 3913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:20.994516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.21.181.66:45447)
0812 07:10:20.994637 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:20.994716 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260812 07:10:20.999013 2765 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 176509da617f4f9f95ed259d9f9766a1: Sending DeleteTablet for 3 replicas of tablet bfb48cecd8464fedbff9f68eeb3ddc07
I20260812 07:10:20.999979 3293 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:20 UTC) from {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:60246
I20260812 07:10:21.000211 3292 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:20 UTC) from {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:60246
I20260812 07:10:21.000422 3569 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:20 UTC) from {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:58882
I20260812 07:10:21.000743 3949 tablet_replica.cc:330] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: stopping tablet replica
I20260812 07:10:21.000829 3568 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:20 UTC) from {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:58882
I20260812 07:10:21.000905 3949 raft_consensus.cc:2229] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:21.000998 2980 tablet_service.cc:1514] Processing DeleteTablet for tablet bfb48cecd8464fedbff9f68eeb3ddc07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:20 UTC) from {username='oryx', principal='oryx/127.21.181.124@KRBTEST.COM'} at 127.0.0.1:59490
I20260812 07:10:21.001264 3949 raft_consensus.cc:2258] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:21.001394 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 2699
I20260812 07:10:21.002252 3949 ts_tablet_manager.cc:1905] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:21.009291 3949 ts_tablet_manager.cc:1918] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:21.009443 3949 log.cc:1196] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/bfb48cecd8464fedbff9f68eeb3ddc07
I20260812 07:10:21.009927 3949 ts_tablet_manager.cc:1939] T bfb48cecd8464fedbff9f68eeb3ddc07 P 1050870805f441d287bd01e27acedd9a: Deleting consensus metadata
I20260812 07:10:21.011006 2745 catalog_manager.cc:4841] TS 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441): tablet bfb48cecd8464fedbff9f68eeb3ddc07 (table test-table [id=e9014c0c46ad404bb5193b80187c99f8]) successfully deleted
W20260812 07:10:21.011456 2745 catalog_manager.cc:4808] TS 1050870805f441d287bd01e27acedd9a (127.21.181.66:39441): delete failed for tablet bfb48cecd8464fedbff9f68eeb3ddc07 because the tablet was not found. No further retry: Not found: Tablet not found: bfb48cecd8464fedbff9f68eeb3ddc07
I20260812 07:10:21.012914 3952 tablet_replica.cc:330] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: stopping tablet replica
I20260812 07:10:21.013237 3952 raft_consensus.cc:2229] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:21.013648 3952 raft_consensus.cc:2258] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:21.014292 3951 tablet_replica.cc:330] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: stopping tablet replica
I20260812 07:10:21.014479 3951 raft_consensus.cc:2229] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:21.014773 3951 raft_consensus.cc:2258] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:21.015245 3952 ts_tablet_manager.cc:1905] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:21.015671 3951 ts_tablet_manager.cc:1905] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:21.021795 3951 ts_tablet_manager.cc:1918] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:21.021800 3952 ts_tablet_manager.cc:1918] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:21.021925 3951 log.cc:1196] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/bfb48cecd8464fedbff9f68eeb3ddc07
I20260812 07:10:21.021934 3952 log.cc:1196] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/bfb48cecd8464fedbff9f68eeb3ddc07
I20260812 07:10:21.022365 3952 ts_tablet_manager.cc:1939] T bfb48cecd8464fedbff9f68eeb3ddc07 P c883c5578a264138870f8efc4d129bd2: Deleting consensus metadata
I20260812 07:10:21.022365 3951 ts_tablet_manager.cc:1939] T bfb48cecd8464fedbff9f68eeb3ddc07 P c1d239dc082f41a4ae5eafce924400f3: Deleting consensus metadata
I20260812 07:10:21.023468 2741 catalog_manager.cc:4841] TS c883c5578a264138870f8efc4d129bd2 (127.21.181.65:44169): tablet bfb48cecd8464fedbff9f68eeb3ddc07 (table test-table [id=e9014c0c46ad404bb5193b80187c99f8]) successfully deleted
I20260812 07:10:21.023510 2740 catalog_manager.cc:4841] TS c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483): tablet bfb48cecd8464fedbff9f68eeb3ddc07 (table test-table [id=e9014c0c46ad404bb5193b80187c99f8]) successfully deleted
W20260812 07:10:21.024012 2740 catalog_manager.cc:4808] TS c1d239dc082f41a4ae5eafce924400f3 (127.21.181.67:38483): delete failed for tablet bfb48cecd8464fedbff9f68eeb3ddc07 because the tablet was not found. No further retry: Not found: Tablet not found: bfb48cecd8464fedbff9f68eeb3ddc07
I20260812 07:10:21.026275 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:21.002974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59492 (local address 127.21.181.65:44169)
0812 07:10:21.003161 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:21.003168 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:21.013603 (+ 10435us) server_negotiation.cc:407] Connection header received
0812 07:10:21.013774 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:21.013779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:21.013861 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:21.013967 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:21.015727 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:21.016539 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:21.017706 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:21.018149 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:21.018255 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:21.020698 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:21.020712 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:21.020727 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:21.020768 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:21.024014 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:21.024654 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:21.024662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:21.024664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:21.024796 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:21.025076 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:21.025079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:21.025081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:21.025396 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:21.025609 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:21.025806 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 07:10:21.025970 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 07:10:21.026907 3114 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:21.116701 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 2699
I20260812 07:10:21.143795 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 3165
I20260812 07:10:21.160429 3892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:21.144554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.21.181.66:39441)
0812 07:10:21.144780 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:21.144785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:21.144828 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:10:21.144967 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:21.144971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:21.145040 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:21.145155 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:21.146445 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:21.147199 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:21.148472 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:21.148920 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:21.149029 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:21.151219 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:21.151235 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:21.151251 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:21.151291 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:21.157988 (+ 6697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:21.158645 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:21.158653 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:21.158656 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:21.158791 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:21.159095 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:21.159098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:21.159100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:21.159455 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:21.159695 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:21.159880 (+ 185us) server_negotiation.cc:299] Negotiation successful
0812 07:10:21.160055 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:10:21.161073 3405 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:21.250913 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:21.250478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45586)
0812 07:10:21.250664 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:21.250761 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:10:21.251983 3893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:21.251608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:41905 (local address 127.0.0.1:45600)
0812 07:10:21.251761 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:21.251855 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:41905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:10:21.280578 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 3165
I20260812 07:10:21.308766 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 3446
I20260812 07:10:21.321379 3891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:21.309523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58884 (local address 127.21.181.67:38483)
0812 07:10:21.309717 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:21.309721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:21.309741 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:10:21.309878 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:21.309882 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:21.309955 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:21.310038 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:21.311367 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:21.312138 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:21.313268 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:21.313676 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:21.313761 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:21.316071 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:21.316178 (+ 107us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:21.316201 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:21.316240 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:21.318933 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:21.319492 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:21.319498 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:21.319500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:21.319621 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:21.319918 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:21.319923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:21.319926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:21.320357 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:21.320587 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:21.320854 (+ 267us) server_negotiation.cc:299] Negotiation successful
0812 07:10:21.321072 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:10:21.321995 3687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:21.413053 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 3446
I20260812 07:10:21.440122 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 2075
I20260812 07:10:21.441140 2412 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:21.556339 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 2075
I20260812 07:10:21.581346 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 2274
I20260812 07:10:21.582393 2826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:21.702633 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 2274
2026-08-12T07:10:21Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9525 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:21 dist-test-slave-lgr5 krb5kdc[3970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:21 dist-test-slave-lgr5 krb5kdc[3970](info): set up 2 sockets
Aug 12 07:10:21 dist-test-slave-lgr5 krb5kdc[3970](info): commencing operation
krb5kdc: starting...
W20260812 07:10:22.943339 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:10:23 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab.
Entry for principal oryx/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab.
I20260812 07:10:23.156558 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43791
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab
--principal=oryx/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:43791
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:23.467383 3990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:23.467646 3990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:23.467774 3990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:23.474705 3990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:23.474802 3990 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260812 07:10:23.474871 3990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:23.474946 3990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:23.475011 3990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:23.475107 3990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:23.484537 3990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:43791
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.126.keytab
--principal=oryx/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:23.485637 3990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:23.486872 3990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:23.497627 3997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:23.497563 3999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:23.497730 4002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:23.498616 3990 server_base.cc:1034] running on GCE node
I20260812 07:10:23.499296 3990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:23.500553 3990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:23.501778 3990 hybrid_clock.cc:648] HybridClock initialized: now 1786518623501718 us; error 24 us; skew 500 ppm
I20260812 07:10:23.502075 3990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:23 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518623, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:23.505039 3990 init.cc:377] Logged in from keytab as oryx/127.21.181.126@KRBTEST.COM (short username oryx)
I20260812 07:10:23.506362 3990 webserver.cc:460] Webserver started at http://127.21.181.126:46397/ using document root <none> and password file <none>
I20260812 07:10:23.507007 3990 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:23.507107 3990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:23.507404 3990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:23.510201 3990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "aced4c0700214bf6a5ff3870b8debb38"
format_stamp: "Formatted at 2026-08-12 07:10:23 on dist-test-slave-lgr5"
server_key: "c159f9150ab77b1dc58f16732e91f929"
server_key_iv: "2f6a37e4fa8fa9f7fc91076ca9c716d5"
server_key_version: "encryptionkey@0"
I20260812 07:10:23.510954 3990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "aced4c0700214bf6a5ff3870b8debb38"
format_stamp: "Formatted at 2026-08-12 07:10:23 on dist-test-slave-lgr5"
server_key: "c159f9150ab77b1dc58f16732e91f929"
server_key_iv: "2f6a37e4fa8fa9f7fc91076ca9c716d5"
server_key_version: "encryptionkey@0"
I20260812 07:10:23.515708 3990 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260812 07:10:23.518886 4012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:23.520134 3990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:23.520311 3990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "aced4c0700214bf6a5ff3870b8debb38"
format_stamp: "Formatted at 2026-08-12 07:10:23 on dist-test-slave-lgr5"
server_key: "c159f9150ab77b1dc58f16732e91f929"
server_key_iv: "2f6a37e4fa8fa9f7fc91076ca9c716d5"
server_key_version: "encryptionkey@0"
I20260812 07:10:23.520503 3990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:23.532022 3990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:23.535859 3990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:23.536121 3990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:23.555516 3990 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:43791
I20260812 07:10:23.555538 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:43791 every 8 connection(s)
I20260812 07:10:23.556797 3990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:23.560220 4128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:23.564311 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 3990
I20260812 07:10:23.564509 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:23.565339 22229 external_mini_cluster.cc:1428] Setting key eb73d33f209d5137efa53c5904bbd303
I20260812 07:10:23.571288 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Bootstrap starting.
I20260812 07:10:23.574677 4128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:23.575766 4128 log.cc:824] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:23.578190 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: No bootstrap required, opened a new log
Aug 12 07:10:23 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:23.585582 4128 raft_consensus.cc:348] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aced4c0700214bf6a5ff3870b8debb38" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43791 } }
I20260812 07:10:23.585908 4128 raft_consensus.cc:374] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:23.586018 4128 raft_consensus.cc:729] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aced4c0700214bf6a5ff3870b8debb38, State: Initialized, Role: FOLLOWER
I20260812 07:10:23.586656 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:23.567193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.21.181.126:43791)
0812 07:10:23.567897 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:23.567908 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:23.567981 (+ 73us) server_negotiation.cc:407] Connection header received
0812 07:10:23.569829 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:23.569850 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:23.570201 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:23.570482 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:23.572215 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:23.573527 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:23.575222 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:23.575864 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:23.576047 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:23.579846 (+ 3799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:23.579856 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:23.579869 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:23.579910 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:23.582807 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:23.583481 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:23.583487 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:23.583493 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:23.583650 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:23.584001 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:23.584005 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:23.584007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:23.584558 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:23.584873 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:23.585285 (+ 412us) server_negotiation.cc:299] Negotiation successful
0812 07:10:23.585652 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":322,"threads_started":1}
I20260812 07:10:23.586581 4128 consensus_queue.cc:260] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aced4c0700214bf6a5ff3870b8debb38" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43791 } }
I20260812 07:10:23.586805 4128 raft_consensus.cc:388] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:23.586937 4128 raft_consensus.cc:482] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:23.587111 4128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:23.589365 4128 raft_consensus.cc:504] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aced4c0700214bf6a5ff3870b8debb38" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43791 } }
I20260812 07:10:23.589833 4128 leader_election.cc:302] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: aced4c0700214bf6a5ff3870b8debb38; no voters:
I20260812 07:10:23.590329 4128 leader_election.cc:288] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:23.590440 4135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:23.591060 4135 raft_consensus.cc:686] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [term 1 LEADER]: Becoming Leader. State: Replica: aced4c0700214bf6a5ff3870b8debb38, State: Running, Role: LEADER
I20260812 07:10:23.591553 4135 consensus_queue.cc:237] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aced4c0700214bf6a5ff3870b8debb38" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43791 } }
I20260812 07:10:23.592180 4128 sys_catalog.cc:564] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:23.594244 4136 sys_catalog.cc:455] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "aced4c0700214bf6a5ff3870b8debb38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aced4c0700214bf6a5ff3870b8debb38" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43791 } } }
I20260812 07:10:23.594633 4136 sys_catalog.cc:458] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:23.596271 4137 sys_catalog.cc:455] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aced4c0700214bf6a5ff3870b8debb38. Latest consensus state: current_term: 1 leader_uuid: "aced4c0700214bf6a5ff3870b8debb38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "aced4c0700214bf6a5ff3870b8debb38" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43791 } } }
I20260812 07:10:23.596628 4137 sys_catalog.cc:458] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:23.599830 4150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:23.603662 4150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:23.610707 4150 catalog_manager.cc:1347] Generated new cluster ID: d65fa8f7ffc14149bbe0add342d093b0
I20260812 07:10:23.610803 4150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:23.636497 4150 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:23.637660 4150 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:23.643299 4150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Generated new TSK 0
I20260812 07:10:23.644035 4150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab.
Entry for principal oryx/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab.
I20260812 07:10:23.703881 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43791
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:24.020300 4174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:24.020579 4174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:24.020730 4174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:24.027757 4174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:24.027872 4174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:24.027997 4174 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:10:24.028121 4174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:24.037436 4174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:24.038563 4174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:24.039798 4174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:24.051860 4183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:24.051937 4182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:24.051900 4189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:24.052794 4174 server_base.cc:1034] running on GCE node
I20260812 07:10:24.053336 4174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:24.054159 4174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:24.055364 4174 hybrid_clock.cc:648] HybridClock initialized: now 1786518624055303 us; error 38 us; skew 500 ppm
I20260812 07:10:24.055655 4174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:24 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518624, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:24.058523 4174 init.cc:377] Logged in from keytab as oryx/127.21.181.65@KRBTEST.COM (short username oryx)
I20260812 07:10:24.059790 4174 webserver.cc:460] Webserver started at http://127.21.181.65:41755/ using document root <none> and password file <none>
I20260812 07:10:24.060463 4174 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:24.060570 4174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:24.060853 4174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:24.063498 4174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "dce88a0bdedb4489b8499b2b44abe68d"
format_stamp: "Formatted at 2026-08-12 07:10:24 on dist-test-slave-lgr5"
server_key: "5045ba9f932581f586b46464617b64b8"
server_key_iv: "8551569bee3ba292b2a1b28e8d65187d"
server_key_version: "encryptionkey@0"
I20260812 07:10:24.064304 4174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "dce88a0bdedb4489b8499b2b44abe68d"
format_stamp: "Formatted at 2026-08-12 07:10:24 on dist-test-slave-lgr5"
server_key: "5045ba9f932581f586b46464617b64b8"
server_key_iv: "8551569bee3ba292b2a1b28e8d65187d"
server_key_version: "encryptionkey@0"
I20260812 07:10:24.068931 4174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:10:24.071997 4197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:24.073185 4174 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:10:24.073329 4174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "dce88a0bdedb4489b8499b2b44abe68d"
format_stamp: "Formatted at 2026-08-12 07:10:24 on dist-test-slave-lgr5"
server_key: "5045ba9f932581f586b46464617b64b8"
server_key_iv: "8551569bee3ba292b2a1b28e8d65187d"
server_key_version: "encryptionkey@0"
I20260812 07:10:24.073514 4174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:24.087281 4174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:24.090742 4174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:24.090976 4174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:24.093595 4174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:24.093698 4174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:24.093863 4174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:24.093940 4174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:10:24 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518624, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:24.116918 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.093873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:56207 (local address 127.21.181.126:43791)
0812 07:10:24.096232 (+ 2359us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:24.096236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:24.096254 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:24.098309 (+ 2055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:24.098313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:24.098378 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:24.098486 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:24.100704 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.101556 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.104345 (+ 2789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.104919 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.105061 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.109876 (+ 4815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:24.109883 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:24.109888 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:24.109928 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:24.112956 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.113873 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.113879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.113883 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.114038 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.114528 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.114533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.114537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.114840 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:24.115011 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:24.116275 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0812 07:10:24.116521 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2264,"thread_start_us":2129,"threads_started":1}
I20260812 07:10:24.117414 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.094494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.21.181.65:56207)
0812 07:10:24.095435 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:24.095482 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:24.097999 (+ 2517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:24.098759 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:24.098770 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:24.099561 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:24.100485 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.100529 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.101738 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.101744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.104187 (+ 2443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.104206 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.105185 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.105193 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.105477 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.106257 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:24.106292 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:24.109621 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:24.113094 (+ 3473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.113103 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.113120 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.113727 (+ 607us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.114185 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.114190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.114194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.114400 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.115112 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:24.115120 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:24.115453 (+ 333us) client_negotiation.cc:769] Sending connection context
0812 07:10:24.115778 (+ 325us) client_negotiation.cc:240] Negotiation successful
0812 07:10:24.116322 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":369,"threads_started":1}
I20260812 07:10:24.137238 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.122423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.21.181.65:41149)
0812 07:10:24.123091 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:24.123115 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:24.123311 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:24.123735 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:24.123739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:24.124446 (+ 707us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:24.124975 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.124993 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.126445 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.126449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.127482 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.127493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.128505 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.128521 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.128776 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.129475 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:24.129501 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:24.130747 (+ 1246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:24.134184 (+ 3437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.134188 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.134196 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.134727 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.135178 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.135183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.135186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.135373 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.135954 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:24.135959 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:24.136232 (+ 273us) client_negotiation.cc:769] Sending connection context
0812 07:10:24.136450 (+ 218us) client_negotiation.cc:240] Negotiation successful
0812 07:10:24.136770 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":357,"threads_started":1}
I20260812 07:10:24.137923 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.122403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:41149 (local address 127.21.181.126:43791)
0812 07:10:24.122574 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:24.122580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:24.123201 (+ 621us) server_negotiation.cc:407] Connection header received
0812 07:10:24.123480 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:24.123486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:24.123587 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:24.123686 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:24.125138 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.126288 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.127601 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.128291 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.128431 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.130980 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:24.130987 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:24.130991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:24.131028 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:24.134024 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.134889 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.134894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.134898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.135025 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.135484 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.135489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.135493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.135852 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:24.135987 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:24.137536 (+ 1549us) server_negotiation.cc:299] Negotiation successful
0812 07:10:24.137714 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:10:24.139168 4174 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:43827
I20260812 07:10:24.139191 4423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:43827 every 8 connection(s)
I20260812 07:10:24.140432 4174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:24.142367 4424 heartbeater.cc:347] Connected to a master server at 127.21.181.126:43791
I20260812 07:10:24.142597 4424 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:24.143190 4424 heartbeater.cc:511] Master 127.21.181.126:43791 requested a full tablet report, sending...
I20260812 07:10:24.145115 4048 ts_manager.cc:194] Registered new tserver with Master: dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827)
I20260812 07:10:24.145608 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 4174
I20260812 07:10:24.145792 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:24.146404 22229 external_mini_cluster.cc:1428] Setting key 7a6f90b5b90fabdfac9e4e4e4b514e92
I20260812 07:10:24.146675 4048 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:56207
WARNING: no policy specified for oryx/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab.
Entry for principal oryx/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab.
I20260812 07:10:24.197100 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43791
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:24.517235 4431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:24.517491 4431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:24.517627 4431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:24.524830 4431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:24.524925 4431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:24.525089 4431 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:10:24.525192 4431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:24.534260 4431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:24.535375 4431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:24.536618 4431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:24.548815 4439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:24.548774 4440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:24.548847 4442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:24.549726 4431 server_base.cc:1034] running on GCE node
I20260812 07:10:24.550278 4431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:24.551087 4431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:24.552311 4431 hybrid_clock.cc:648] HybridClock initialized: now 1786518624552244 us; error 41 us; skew 500 ppm
I20260812 07:10:24.552567 4431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:24 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518624, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:24.555652 4431 init.cc:377] Logged in from keytab as oryx/127.21.181.66@KRBTEST.COM (short username oryx)
I20260812 07:10:24.557257 4431 webserver.cc:460] Webserver started at http://127.21.181.66:37557/ using document root <none> and password file <none>
I20260812 07:10:24.557893 4431 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:24.558003 4431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:24.558295 4431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:24.560995 4431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "b002ca2e965a45f58223bbeef4d192d0"
format_stamp: "Formatted at 2026-08-12 07:10:24 on dist-test-slave-lgr5"
server_key: "57e67cd1f7118cfd6f9e5b561c374eed"
server_key_iv: "9c632ce2897eefcff60eeea5ea340b65"
server_key_version: "encryptionkey@0"
I20260812 07:10:24.561738 4431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "b002ca2e965a45f58223bbeef4d192d0"
format_stamp: "Formatted at 2026-08-12 07:10:24 on dist-test-slave-lgr5"
server_key: "57e67cd1f7118cfd6f9e5b561c374eed"
server_key_iv: "9c632ce2897eefcff60eeea5ea340b65"
server_key_version: "encryptionkey@0"
I20260812 07:10:24.566586 4431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:10:24.569761 4460 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:24.570896 4431 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:10:24.571053 4431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "b002ca2e965a45f58223bbeef4d192d0"
format_stamp: "Formatted at 2026-08-12 07:10:24 on dist-test-slave-lgr5"
server_key: "57e67cd1f7118cfd6f9e5b561c374eed"
server_key_iv: "9c632ce2897eefcff60eeea5ea340b65"
server_key_version: "encryptionkey@0"
I20260812 07:10:24.571238 4431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:24.597679 4431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:24.601660 4431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:24.601869 4431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:24.604254 4431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:24.604338 4431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:24.604465 4431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:24.604537 4431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 12 07:10:24 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518624, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:24.626204 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.604789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:49799 (local address 127.21.181.126:43791)
0812 07:10:24.604979 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:24.604985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:24.606364 (+ 1379us) server_negotiation.cc:407] Connection header received
0812 07:10:24.609029 (+ 2665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:24.609035 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:24.609135 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:24.609239 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:24.611301 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.612463 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.615299 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.615837 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.615966 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.619723 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:24.619728 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:24.619731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:24.619763 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:24.622473 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.623340 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.623346 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.623351 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.623541 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.624129 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.624135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.624139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.624524 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:24.624704 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:24.625582 (+ 878us) server_negotiation.cc:299] Negotiation successful
0812 07:10:24.625828 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:10:24.626945 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.605313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.21.181.66:49799)
0812 07:10:24.606179 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:24.606227 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:24.608713 (+ 2486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:24.609521 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:24.609532 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:24.610278 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:24.611068 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.611097 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.612682 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.612691 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.615129 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.615147 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.616126 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.616139 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.616437 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.617118 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:24.617141 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:24.619525 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:24.622616 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.622627 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.622642 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.623214 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.623692 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.623699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.623703 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.623983 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.624652 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:24.624659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:24.625143 (+ 484us) client_negotiation.cc:769] Sending connection context
0812 07:10:24.625469 (+ 326us) client_negotiation.cc:240] Negotiation successful
0812 07:10:24.625868 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":331,"threads_started":1}
I20260812 07:10:24.645491 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.631920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:58625 (local address 127.21.181.126:43791)
0812 07:10:24.632131 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:24.632136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:24.632707 (+ 571us) server_negotiation.cc:407] Connection header received
0812 07:10:24.632859 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:24.632862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:24.632916 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:24.632986 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:24.634103 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.634808 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.636977 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.637476 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.637597 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.639794 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:24.639798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:24.639801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:24.639829 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:24.642522 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.643323 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.643327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.643329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.643470 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:24.643858 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:24.643863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:24.643866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:24.644343 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:24.644503 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:24.644972 (+ 469us) server_negotiation.cc:299] Negotiation successful
0812 07:10:24.645183 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:10:24.645610 4614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:24.631928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.21.181.66:58625)
0812 07:10:24.632604 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:24.632624 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:24.632760 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:24.633027 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:24.633032 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:24.633618 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:24.633983 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.633996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.634972 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.634977 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.636817 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:24.636837 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:24.637805 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:24.637810 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:24.637985 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:24.638604 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:24.638621 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:24.639580 (+ 959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:24.642659 (+ 3079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.642664 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.642668 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.643214 (+ 546us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.643582 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:24.643588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:24.643592 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:24.643719 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:24.644514 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:24.644519 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:24.644711 (+ 192us) client_negotiation.cc:769] Sending connection context
0812 07:10:24.644875 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 07:10:24.645177 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":367,"threads_started":1}
I20260812 07:10:24.652573 4431 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:36257
I20260812 07:10:24.652594 4714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:36257 every 8 connection(s)
I20260812 07:10:24.653664 4431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:24.655440 4715 heartbeater.cc:347] Connected to a master server at 127.21.181.126:43791
I20260812 07:10:24.655632 4715 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:24.656216 4715 heartbeater.cc:511] Master 127.21.181.126:43791 requested a full tablet report, sending...
I20260812 07:10:24.657414 4043 ts_manager.cc:194] Registered new tserver with Master: b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:24.658212 4043 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:49799
I20260812 07:10:24.659349 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 4431
I20260812 07:10:24.659548 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:24.660137 22229 external_mini_cluster.cc:1428] Setting key 7dcc56fbdd3ba6d745b4717c361d64c7
WARNING: no policy specified for oryx/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab.
Entry for principal oryx/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab.
I20260812 07:10:24.708936 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:43791
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:25.022521 4728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:25.022778 4728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:25.022919 4728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:25.029981 4728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:25.030090 4728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:25.030213 4728 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260812 07:10:25.030323 4728 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:25.039994 4728 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.21.181.126:43791
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:25.041095 4728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:25.042323 4728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:25.054409 4743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:25.054351 4738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:25.054441 4739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:25.055519 4728 server_base.cc:1034] running on GCE node
I20260812 07:10:25.056102 4728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:25.056927 4728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:25.058138 4728 hybrid_clock.cc:648] HybridClock initialized: now 1786518625058076 us; error 39 us; skew 500 ppm
I20260812 07:10:25.058408 4728 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518625, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:25.061319 4728 init.cc:377] Logged in from keytab as oryx/127.21.181.67@KRBTEST.COM (short username oryx)
I20260812 07:10:25.062563 4728 webserver.cc:460] Webserver started at http://127.21.181.67:35547/ using document root <none> and password file <none>
I20260812 07:10:25.063174 4728 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:25.063256 4728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:25.063515 4728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:25.066251 4728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "a108ecfef49a4ced975671472692196b"
format_stamp: "Formatted at 2026-08-12 07:10:25 on dist-test-slave-lgr5"
server_key: "e2454e174a363b45df991d7b72eaba4d"
server_key_iv: "a7b144e5258d038888cf22fe06864a1f"
server_key_version: "encryptionkey@0"
I20260812 07:10:25.067044 4728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "a108ecfef49a4ced975671472692196b"
format_stamp: "Formatted at 2026-08-12 07:10:25 on dist-test-slave-lgr5"
server_key: "e2454e174a363b45df991d7b72eaba4d"
server_key_iv: "a7b144e5258d038888cf22fe06864a1f"
server_key_version: "encryptionkey@0"
I20260812 07:10:25.071892 4728 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:10:25.075229 4757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:25.076505 4728 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:25.076658 4728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "a108ecfef49a4ced975671472692196b"
format_stamp: "Formatted at 2026-08-12 07:10:25 on dist-test-slave-lgr5"
server_key: "e2454e174a363b45df991d7b72eaba4d"
server_key_iv: "a7b144e5258d038888cf22fe06864a1f"
server_key_version: "encryptionkey@0"
I20260812 07:10:25.076846 4728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:25.094795 4728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:25.098522 4728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:25.098747 4728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:25.101301 4728 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:25.101393 4728 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:25.101523 4728 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:25.101627 4728 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518625, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:25.125039 4768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.102162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.21.181.67:34403)
0812 07:10:25.103000 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.103055 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.105085 (+ 2030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.105821 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.105836 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.106589 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:25.107443 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.107472 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.109110 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.109116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.111021 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.111036 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.111963 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.111971 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.112274 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.112998 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:25.113034 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:25.116264 (+ 3230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:25.119971 (+ 3707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:25.119981 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:25.120000 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:25.120744 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:25.121215 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:25.121223 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:25.121227 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:25.121448 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:25.122147 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:25.122155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:25.122575 (+ 420us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.122915 (+ 340us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.123426 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":350,"threads_started":1}
I20260812 07:10:25.125785 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.101671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:34403 (local address 127.21.181.126:43791)
0812 07:10:25.101863 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.101869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.103187 (+ 1318us) server_negotiation.cc:407] Connection header received
0812 07:10:25.105343 (+ 2156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.105348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.105446 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.105541 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:25.107686 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.108890 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.111161 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.111727 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.111841 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.116471 (+ 4630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:25.116484 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:25.116488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:25.116521 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:25.119838 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.120880 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.120885 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.120888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.121062 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.121608 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.121614 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.121617 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.121988 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:25.122153 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.125199 (+ 3046us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.125447 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:10:25.143188 4728 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:39191
I20260812 07:10:25.143198 4989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:39191 every 8 connection(s)
I20260812 07:10:25.144600 4728 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:25.146737 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.130909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:50013 (local address 127.21.181.126:43791)
0812 07:10:25.131086 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.131091 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.132611 (+ 1520us) server_negotiation.cc:407] Connection header received
0812 07:10:25.132758 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.132763 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.132850 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.132940 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:25.134258 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.135060 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.137316 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.137897 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.138033 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.140517 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:25.140524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:25.140528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:25.140566 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:25.143864 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.144679 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.144684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.144687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.144835 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.145230 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.145235 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.145238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.145593 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:25.145728 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.146194 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.146412 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:10:25.146822 4990 heartbeater.cc:347] Connected to a master server at 127.21.181.126:43791
I20260812 07:10:25.146845 4943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.130979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.21.181.67:50013)
0812 07:10:25.132515 (+ 1536us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.132532 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.132653 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.133026 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.133032 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.133682 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:25.134120 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.134135 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.135223 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.135228 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.137111 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.137128 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.138115 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.138124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.138314 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.139003 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:25.139025 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:25.140259 (+ 1234us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:25.144003 (+ 3744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:25.144008 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:25.144011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:25.144519 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:25.144940 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:25.144944 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:25.144947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:25.145094 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:25.145708 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:25.145713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:25.145915 (+ 202us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.146072 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.146371 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1402,"thread_start_us":1274,"threads_started":1}
I20260812 07:10:25.147089 4990 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:25.147647 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 4728
I20260812 07:10:25.147836 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:25.147927 4990 heartbeater.cc:511] Master 127.21.181.126:43791 requested a full tablet report, sending...
I20260812 07:10:25.148454 22229 external_mini_cluster.cc:1428] Setting key c86f643d601c116ff5b3375158c09067
I20260812 07:10:25.149338 4424 heartbeater.cc:503] Master 127.21.181.126:43791 was elected leader, sending a full tablet report...
I20260812 07:10:25.149501 4043 ts_manager.cc:194] Registered new tserver with Master: a108ecfef49a4ced975671472692196b (127.21.181.67:39191)
I20260812 07:10:25.150560 4043 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:34403
I20260812 07:10:25.151719 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:10:25.205382 4210 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:25.195643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48204 (local address 127.21.181.126:43791)
0812 07:10:25.195871 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.195877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.196003 (+ 126us) server_negotiation.cc:407] Connection header received
0812 07:10:25.196222 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.196227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.196294 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.196380 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:25.197673 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.198412 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.199629 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.200048 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.200179 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.205170 (+ 4991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48204: BlockingRecv error: recv got EOF from 127.0.0.1:48204 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:25.219995 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.208339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.21.181.126:43791)
0812 07:10:25.208475 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.208480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.208805 (+ 325us) server_negotiation.cc:407] Connection header received
0812 07:10:25.208917 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.208921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.208996 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.209071 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:25.210083 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.210721 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.211815 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.212273 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.212356 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.215647 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:25.215651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:25.215654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:25.215685 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:25.217901 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.218458 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.218461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.218463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.218604 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.218883 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.218886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.218888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.219168 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:25.219383 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.219574 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.219759 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260812 07:10:25.223594 4048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:25.230466 4048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:25.252686 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.241896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.21.181.65:43827)
0812 07:10:25.243844 (+ 1948us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.243852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.244536 (+ 684us) server_negotiation.cc:407] Connection header received
0812 07:10:25.244960 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.244968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.245204 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.245379 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.246504 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.247858 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.250606 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.251559 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.251684 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.251894 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.252237 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.252407 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1789,"thread_start_us":1649,"threads_started":1}
I20260812 07:10:25.252739 5024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.241619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43827 (local address 127.0.0.1:56656)
0812 07:10:25.244387 (+ 2768us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.244435 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.244700 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.245685 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.245691 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.245725 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.246345 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.246355 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.247982 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.247987 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.250449 (+ 2462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.250461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.251772 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.251779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.251972 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.252013 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.252149 (+ 136us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.252328 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2493,"mutex_wait_us":153,"spinlock_wait_cycles":2944,"thread_start_us":2530,"threads_started":1}
I20260812 07:10:25.254616 5029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.245295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36257 (local address 127.0.0.1:57162)
0812 07:10:25.246026 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.246042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.246149 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.247394 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.247399 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.247423 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.247755 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.247765 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.249471 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.249476 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.251703 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.251718 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.253817 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.253823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.253979 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.254012 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.254100 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.254226 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":71,"thread_start_us":407,"threads_started":1}
I20260812 07:10:25.254933 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.241865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:39191 (local address 127.0.0.1:57818)
0812 07:10:25.244422 (+ 2557us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.244439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.244700 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.245709 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.245715 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.245742 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.246308 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.246322 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.248041 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.248045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.251984 (+ 3939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.252008 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.254187 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.254192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.254306 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.254331 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.254406 (+ 75us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.254539 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2436,"mutex_wait_us":31,"spinlock_wait_cycles":8064,"thread_start_us":2018,"threads_started":1}
I20260812 07:10:25.255008 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.242568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57818 (local address 127.21.181.67:39191)
0812 07:10:25.243384 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.243394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.244534 (+ 1140us) server_negotiation.cc:407] Connection header received
0812 07:10:25.244941 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.244949 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.245217 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.245373 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.246574 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.247866 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.252154 (+ 4288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.253659 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.253836 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.254111 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.254478 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.254706 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":423,"threads_started":1}
I20260812 07:10:25.255416 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.244961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57162 (local address 127.21.181.66:36257)
0812 07:10:25.246847 (+ 1886us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.246855 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.246883 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:10:25.246971 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.246990 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.247205 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.247376 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.247937 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.249316 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.251889 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.253344 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.253473 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.253670 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.254951 (+ 1281us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.255137 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1725,"thread_start_us":1586,"threads_started":1}
I20260812 07:10:25.256953 4290 tablet_service.cc:1467] Processing CreateTablet for tablet 97dcc822eb8a4d6fab30efa6ff18de65 (DEFAULT_TABLE table=test-table [id=6ef01e9758fb4866a4e0cb72c51a3b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:25.258232 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97dcc822eb8a4d6fab30efa6ff18de65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.258394 4847 tablet_service.cc:1467] Processing CreateTablet for tablet 97dcc822eb8a4d6fab30efa6ff18de65 (DEFAULT_TABLE table=test-table [id=6ef01e9758fb4866a4e0cb72c51a3b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:25.258584 4566 tablet_service.cc:1467] Processing CreateTablet for tablet 97dcc822eb8a4d6fab30efa6ff18de65 (DEFAULT_TABLE table=test-table [id=6ef01e9758fb4866a4e0cb72c51a3b86]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:25.259778 4566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97dcc822eb8a4d6fab30efa6ff18de65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.259778 4847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97dcc822eb8a4d6fab30efa6ff18de65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.270016 5030 tablet_bootstrap.cc:492] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Bootstrap starting.
I20260812 07:10:25.273890 5030 tablet_bootstrap.cc:654] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.273914 5032 tablet_bootstrap.cc:492] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Bootstrap starting.
I20260812 07:10:25.275413 5030 log.cc:824] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:25.277043 5031 tablet_bootstrap.cc:492] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Bootstrap starting.
I20260812 07:10:25.277609 5032 tablet_bootstrap.cc:654] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.278914 5032 log.cc:824] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:25.279273 5030 tablet_bootstrap.cc:492] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: No bootstrap required, opened a new log
I20260812 07:10:25.279606 5030 ts_tablet_manager.cc:1397] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260812 07:10:25.283690 5031 tablet_bootstrap.cc:654] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.284885 5031 log.cc:824] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:25.286480 5032 tablet_bootstrap.cc:492] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: No bootstrap required, opened a new log
I20260812 07:10:25.286785 5032 ts_tablet_manager.cc:1397] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.005s
I20260812 07:10:25.287961 5031 tablet_bootstrap.cc:492] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: No bootstrap required, opened a new log
I20260812 07:10:25.288286 5031 ts_tablet_manager.cc:1397] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.003s
I20260812 07:10:25.291615 5030 raft_consensus.cc:348] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.291950 5030 raft_consensus.cc:374] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.292073 5030 raft_consensus.cc:729] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce88a0bdedb4489b8499b2b44abe68d, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.292673 5030 consensus_queue.cc:260] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.294332 5030 ts_tablet_manager.cc:1428] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Time spent starting tablet: real 0.015s user 0.010s sys 0.000s
I20260812 07:10:25.297778 5032 raft_consensus.cc:348] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.298231 5032 raft_consensus.cc:374] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.298349 5032 raft_consensus.cc:729] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a108ecfef49a4ced975671472692196b, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.298954 5032 consensus_queue.cc:260] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.299237 5031 raft_consensus.cc:348] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.299678 5031 raft_consensus.cc:374] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.299809 5031 raft_consensus.cc:729] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b002ca2e965a45f58223bbeef4d192d0, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.300055 4990 heartbeater.cc:503] Master 127.21.181.126:43791 was elected leader, sending a full tablet report...
I20260812 07:10:25.300640 5031 consensus_queue.cc:260] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.300879 5032 ts_tablet_manager.cc:1428] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Time spent starting tablet: real 0.014s user 0.013s sys 0.003s
I20260812 07:10:25.301772 4715 heartbeater.cc:503] Master 127.21.181.126:43791 was elected leader, sending a full tablet report...
I20260812 07:10:25.302340 5031 ts_tablet_manager.cc:1428] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
W20260812 07:10:25.393215 4428 tablet.cc:2369] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:10:25.396206 4991 tablet.cc:2369] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:10:25.405444 4716 tablet.cc:2369] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:10:25.435140 5043 raft_consensus.cc:482] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:25.435441 5043 raft_consensus.cc:504] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.437104 5043 leader_election.cc:288] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a108ecfef49a4ced975671472692196b (127.21.181.67:39191), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:25.443643 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.437546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:60099 (local address 127.21.181.67:39191)
0812 07:10:25.437831 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.437838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.438440 (+ 602us) server_negotiation.cc:407] Connection header received
0812 07:10:25.438633 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.438637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.438702 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.438789 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.439418 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.440125 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.441984 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.442728 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.442862 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.442937 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.443265 (+ 328us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.443387 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260812 07:10:25.443673 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.437586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:55545 (local address 127.21.181.66:36257)
0812 07:10:25.437994 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.438000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.438300 (+ 300us) server_negotiation.cc:407] Connection header received
0812 07:10:25.438568 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.438573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.438643 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.438708 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.439356 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.440134 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.441991 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.442741 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.442900 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.442972 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.443310 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.443433 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260812 07:10:25.443763 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.437369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:39191 (local address 127.21.181.65:60099)
0812 07:10:25.438353 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.438367 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.438488 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.438820 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.438825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.438852 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.439252 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.439263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.440260 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.440263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.441837 (+ 1574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.441850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.442938 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.442945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.443105 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.443140 (+ 35us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.443211 (+ 71us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.443320 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":86,"thread_start_us":638,"threads_started":1}
I20260812 07:10:25.443786 5050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.437366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36257 (local address 127.21.181.65:55545)
0812 07:10:25.438179 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.438199 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.438355 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.438755 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.438760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.438785 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.439217 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.439228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.440256 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.440260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.441835 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.441847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.442976 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.442983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.443155 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.443178 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.443269 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.443387 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":66,"thread_start_us":381,"threads_started":1}
I20260812 07:10:25.444742 4627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97dcc822eb8a4d6fab30efa6ff18de65" candidate_uuid: "dce88a0bdedb4489b8499b2b44abe68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0" is_pre_election: true
I20260812 07:10:25.444914 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97dcc822eb8a4d6fab30efa6ff18de65" candidate_uuid: "dce88a0bdedb4489b8499b2b44abe68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a108ecfef49a4ced975671472692196b" is_pre_election: true
I20260812 07:10:25.445201 4627 raft_consensus.cc:2454] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dce88a0bdedb4489b8499b2b44abe68d in term 0.
I20260812 07:10:25.445433 4892 raft_consensus.cc:2454] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dce88a0bdedb4489b8499b2b44abe68d in term 0.
I20260812 07:10:25.445829 4203 leader_election.cc:302] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b002ca2e965a45f58223bbeef4d192d0, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:25.446233 5043 raft_consensus.cc:2781] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:25.446362 5043 raft_consensus.cc:482] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:25.446455 5043 raft_consensus.cc:3037] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.448712 5043 raft_consensus.cc:504] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.449295 5043 leader_election.cc:288] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [CANDIDATE]: Term 1 election: Requested vote from peers a108ecfef49a4ced975671472692196b (127.21.181.67:39191), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:25.449687 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97dcc822eb8a4d6fab30efa6ff18de65" candidate_uuid: "dce88a0bdedb4489b8499b2b44abe68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a108ecfef49a4ced975671472692196b"
I20260812 07:10:25.449782 4627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97dcc822eb8a4d6fab30efa6ff18de65" candidate_uuid: "dce88a0bdedb4489b8499b2b44abe68d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0"
I20260812 07:10:25.449883 4892 raft_consensus.cc:3037] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.449977 4627 raft_consensus.cc:3037] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.452013 4892 raft_consensus.cc:2454] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dce88a0bdedb4489b8499b2b44abe68d in term 1.
I20260812 07:10:25.452142 4627 raft_consensus.cc:2454] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dce88a0bdedb4489b8499b2b44abe68d in term 1.
I20260812 07:10:25.452412 4200 leader_election.cc:302] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:25.452826 5043 raft_consensus.cc:2781] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:25.453217 5043 raft_consensus.cc:686] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 1 LEADER]: Becoming Leader. State: Replica: dce88a0bdedb4489b8499b2b44abe68d, State: Running, Role: LEADER
I20260812 07:10:25.453648 5043 consensus_queue.cc:237] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } }
I20260812 07:10:25.457942 4043 catalog_manager.cc:5495] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d reported cstate change: term changed from 0 to 1, leader changed from <none> to dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65). New cstate: current_term: 1 leader_uuid: "dce88a0bdedb4489b8499b2b44abe68d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } health_report { overall_health: UNKNOWN } } }
I20260812 07:10:25.486088 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.481461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.21.181.65:43827)
0812 07:10:25.481652 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.481657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.481708 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:10:25.481893 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.481897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.481985 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.482114 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:25.482373 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.483062 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.484310 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.484711 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.484810 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.485107 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:25.485600 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:25.485705 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.485745 (+ 40us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.485808 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260812 07:10:25.493685 4892 raft_consensus.cc:1261] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Refusing update from remote peer dce88a0bdedb4489b8499b2b44abe68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:25.493681 4627 raft_consensus.cc:1261] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Refusing update from remote peer dce88a0bdedb4489b8499b2b44abe68d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:25.494627 5054 consensus_queue.cc:1035] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:25.494947 5043 consensus_queue.cc:1035] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:25.505095 5059 mvcc.cc:205] Tried to move back new op lower bound from 7317580290012495872 to 7317580289861640192. Current Snapshot: MvccSnapshot[applied={T|T < 7317580290012495872}]
I20260812 07:10:25.517807 4043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:25.518079 4043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:25.521660 4043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Sending DeleteTablet for 3 replicas of tablet 97dcc822eb8a4d6fab30efa6ff18de65
I20260812 07:10:25.522702 4290 tablet_service.cc:1514] Processing DeleteTablet for tablet 97dcc822eb8a4d6fab30efa6ff18de65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:25 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56656
I20260812 07:10:25.522882 4847 tablet_service.cc:1514] Processing DeleteTablet for tablet 97dcc822eb8a4d6fab30efa6ff18de65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:25 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57818
I20260812 07:10:25.523022 4566 tablet_service.cc:1514] Processing DeleteTablet for tablet 97dcc822eb8a4d6fab30efa6ff18de65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:25 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57162
I20260812 07:10:25.524812 4043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:25.525595 4043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:25.533289 4842 tablet_service.cc:1467] Processing CreateTablet for tablet c80baf852af149b2bf64c5cebf32095c (DEFAULT_TABLE table=test-table [id=7465cc5204474160842e39bbe72bb3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:25.533391 4289 tablet_service.cc:1467] Processing CreateTablet for tablet c80baf852af149b2bf64c5cebf32095c (DEFAULT_TABLE table=test-table [id=7465cc5204474160842e39bbe72bb3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:25.533411 4565 tablet_service.cc:1467] Processing CreateTablet for tablet c80baf852af149b2bf64c5cebf32095c (DEFAULT_TABLE table=test-table [id=7465cc5204474160842e39bbe72bb3dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:25.533882 4842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80baf852af149b2bf64c5cebf32095c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.533893 4289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80baf852af149b2bf64c5cebf32095c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.534092 4565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c80baf852af149b2bf64c5cebf32095c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.540450 5030 tablet_bootstrap.cc:492] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Bootstrap starting.
I20260812 07:10:25.540690 5031 tablet_bootstrap.cc:492] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Bootstrap starting.
I20260812 07:10:25.542172 5030 tablet_bootstrap.cc:654] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.542157 5083 tablet_replica.cc:330] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: stopping tablet replica
I20260812 07:10:25.542371 5031 tablet_bootstrap.cc:654] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.542608 5083 raft_consensus.cc:2229] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:25.543162 5083 raft_consensus.cc:2258] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:25.545348 5083 ts_tablet_manager.cc:1905] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:25.550882 5032 tablet_bootstrap.cc:492] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Bootstrap starting.
I20260812 07:10:25.552112 5083 ts_tablet_manager.cc:1918] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:25.552263 5083 log.cc:1196] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/97dcc822eb8a4d6fab30efa6ff18de65
I20260812 07:10:25.552534 5032 tablet_bootstrap.cc:654] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.552825 5083 ts_tablet_manager.cc:1939] T 97dcc822eb8a4d6fab30efa6ff18de65 P b002ca2e965a45f58223bbeef4d192d0: Deleting consensus metadata
I20260812 07:10:25.554643 4016 catalog_manager.cc:4841] TS b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257): tablet 97dcc822eb8a4d6fab30efa6ff18de65 (table test-table [id=6ef01e9758fb4866a4e0cb72c51a3b86]) successfully deleted
I20260812 07:10:25.557250 5030 tablet_bootstrap.cc:492] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: No bootstrap required, opened a new log
I20260812 07:10:25.557402 5030 ts_tablet_manager.cc:1397] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.005s
I20260812 07:10:25.558302 5081 tablet_replica.cc:330] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: stopping tablet replica
I20260812 07:10:25.558264 5030 raft_consensus.cc:348] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.558490 5030 raft_consensus.cc:374] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.558581 5030 raft_consensus.cc:729] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce88a0bdedb4489b8499b2b44abe68d, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.558693 5081 raft_consensus.cc:2229] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:25.558872 5030 consensus_queue.cc:260] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.559430 5030 ts_tablet_manager.cc:1428] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 07:10:25.560834 5081 raft_consensus.cc:2258] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:25.561590 5080 tablet_replica.cc:330] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: stopping tablet replica
I20260812 07:10:25.561967 5080 raft_consensus.cc:2229] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:25.563799 5080 raft_consensus.cc:2258] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:25.563985 5031 tablet_bootstrap.cc:492] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: No bootstrap required, opened a new log
I20260812 07:10:25.564191 5031 ts_tablet_manager.cc:1397] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.007s
I20260812 07:10:25.565142 5081 ts_tablet_manager.cc:1905] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:25.565266 5031 raft_consensus.cc:348] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.565518 5031 raft_consensus.cc:374] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.565663 5031 raft_consensus.cc:729] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b002ca2e965a45f58223bbeef4d192d0, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.566016 5031 consensus_queue.cc:260] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.566443 5080 ts_tablet_manager.cc:1905] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:25.568506 5031 ts_tablet_manager.cc:1428] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 07:10:25.571660 5081 ts_tablet_manager.cc:1918] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:25.571867 5081 log.cc:1196] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/97dcc822eb8a4d6fab30efa6ff18de65
I20260812 07:10:25.572443 5081 ts_tablet_manager.cc:1939] T 97dcc822eb8a4d6fab30efa6ff18de65 P a108ecfef49a4ced975671472692196b: Deleting consensus metadata
I20260812 07:10:25.574177 4014 catalog_manager.cc:4841] TS a108ecfef49a4ced975671472692196b (127.21.181.67:39191): tablet 97dcc822eb8a4d6fab30efa6ff18de65 (table test-table [id=6ef01e9758fb4866a4e0cb72c51a3b86]) successfully deleted
I20260812 07:10:25.574220 5080 ts_tablet_manager.cc:1918] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:25.574349 5080 log.cc:1196] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/97dcc822eb8a4d6fab30efa6ff18de65
I20260812 07:10:25.575117 5080 ts_tablet_manager.cc:1939] T 97dcc822eb8a4d6fab30efa6ff18de65 P dce88a0bdedb4489b8499b2b44abe68d: Deleting consensus metadata
I20260812 07:10:25.575410 5032 tablet_bootstrap.cc:492] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: No bootstrap required, opened a new log
I20260812 07:10:25.575601 5032 ts_tablet_manager.cc:1397] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.003s
I20260812 07:10:25.576514 5032 raft_consensus.cc:348] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.576687 4016 catalog_manager.cc:4841] TS dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827): tablet 97dcc822eb8a4d6fab30efa6ff18de65 (table test-table [id=6ef01e9758fb4866a4e0cb72c51a3b86]) successfully deleted
I20260812 07:10:25.576728 5032 raft_consensus.cc:374] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.576812 5032 raft_consensus.cc:729] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a108ecfef49a4ced975671472692196b, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.577064 5032 consensus_queue.cc:260] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.579341 5032 ts_tablet_manager.cc:1428] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260812 07:10:25.642550 5044 raft_consensus.cc:482] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:25.642755 5044 raft_consensus.cc:504] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.644502 5044 leader_election.cc:288] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:25.652290 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.645490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:43493 (local address 127.21.181.66:36257)
0812 07:10:25.645663 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.645669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.646237 (+ 568us) server_negotiation.cc:407] Connection header received
0812 07:10:25.646408 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.646413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.646499 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.646575 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.647186 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.648131 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.650457 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.651283 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.651431 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.651518 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.651792 (+ 274us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.651933 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:10:25.652709 5090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.644670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43827 (local address 127.21.181.67:60849)
0812 07:10:25.645154 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.645174 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.645365 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.646006 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.646009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.646032 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.646322 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.646336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.647782 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.647787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.650154 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.650171 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.651816 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.651822 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.651989 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.652020 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.652287 (+ 267us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.652430 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":211,"threads_started":1}
I20260812 07:10:25.652822 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.644785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:60849 (local address 127.21.181.65:43827)
0812 07:10:25.644985 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.644991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.645251 (+ 260us) server_negotiation.cc:407] Connection header received
0812 07:10:25.645566 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.645577 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.645704 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.645771 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.646474 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.647640 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.650371 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.651608 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.651763 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.651837 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.652329 (+ 492us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.652459 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:10:25.653198 5092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.645272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36257 (local address 127.21.181.67:43493)
0812 07:10:25.646114 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.646135 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.646276 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.646603 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.646607 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.646631 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.646931 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.646943 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.648318 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.648323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.650319 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.650330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.651444 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.651450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.651618 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.651642 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.652669 (+ 1027us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.652916 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"thread_start_us":508,"threads_started":1}
I20260812 07:10:25.653637 4328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80baf852af149b2bf64c5cebf32095c" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d" is_pre_election: true
I20260812 07:10:25.654119 4328 raft_consensus.cc:2454] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 0.
I20260812 07:10:25.654850 4760 leader_election.cc:302] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:25.654928 4627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80baf852af149b2bf64c5cebf32095c" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0" is_pre_election: true
I20260812 07:10:25.655174 4627 raft_consensus.cc:2454] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 0.
I20260812 07:10:25.655259 5044 raft_consensus.cc:2781] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:25.655366 5044 raft_consensus.cc:482] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:25.655443 5044 raft_consensus.cc:3037] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.658488 5044 raft_consensus.cc:504] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.659119 5044 leader_election.cc:288] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 election: Requested vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:25.659675 4328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80baf852af149b2bf64c5cebf32095c" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d"
I20260812 07:10:25.659870 4328 raft_consensus.cc:3037] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.659852 4627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c80baf852af149b2bf64c5cebf32095c" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0"
I20260812 07:10:25.660043 4627 raft_consensus.cc:3037] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.662976 4627 raft_consensus.cc:2454] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 1.
I20260812 07:10:25.663092 4328 raft_consensus.cc:2454] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 1.
I20260812 07:10:25.663368 4760 leader_election.cc:302] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, b002ca2e965a45f58223bbeef4d192d0; no voters:
I20260812 07:10:25.663635 5044 raft_consensus.cc:2781] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:25.664141 5044 raft_consensus.cc:686] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 1 LEADER]: Becoming Leader. State: Replica: a108ecfef49a4ced975671472692196b, State: Running, Role: LEADER
I20260812 07:10:25.664796 5044 consensus_queue.cc:237] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.668399 4042 catalog_manager.cc:5495] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b reported cstate change: term changed from 0 to 1, leader changed from <none> to a108ecfef49a4ced975671472692196b (127.21.181.67). New cstate: current_term: 1 leader_uuid: "a108ecfef49a4ced975671472692196b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } health_report { overall_health: HEALTHY } } }
Aug 12 07:10:25 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518623, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.126@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:25.744228 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.730353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:43791 (local address 127.0.0.1:48220)
0812 07:10:25.731002 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.731031 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.731145 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.731778 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.731784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.732549 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:25.732814 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.732825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.733893 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.733897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.734823 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.734837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.735544 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.735555 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.735716 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.736357 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:25.736374 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:25.738783 (+ 2409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:25.741577 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:25.741590 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:25.741597 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:25.742140 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:25.742517 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:25.742522 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:25.742526 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:25.742682 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:25.743166 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:25.743173 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:25.743360 (+ 187us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.743531 (+ 171us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.743800 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":352,"threads_started":1}
I20260812 07:10:25.744300 5111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.730439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48220 (local address 127.21.181.126:43791)
0812 07:10:25.731515 (+ 1076us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.731519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.731535 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:10:25.731574 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.731577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.731626 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.731715 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:25.732966 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.733757 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.734950 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.735377 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.735482 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.738991 (+ 3509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:25.738996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:25.738999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:25.739049 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:25.741454 (+ 2405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.742267 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.742272 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.742274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.742407 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:25.742768 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:25.742771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:25.742773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:25.743050 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:25.743156 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.743608 (+ 452us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.743923 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":984,"thread_start_us":801,"threads_started":1}
I20260812 07:10:25.749516 4042 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:48220:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260812 07:10:25.750543 4042 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:25.757186 4290 tablet_service.cc:1467] Processing CreateTablet for tablet 35c443fa283c49008268ff9fba1f05a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77ee9dbd3d0e4affa3d13e888fa597d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:10:25.757444 4566 tablet_service.cc:1467] Processing CreateTablet for tablet 35c443fa283c49008268ff9fba1f05a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77ee9dbd3d0e4affa3d13e888fa597d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:10:25.757786 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c443fa283c49008268ff9fba1f05a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.758038 4566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c443fa283c49008268ff9fba1f05a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.762971 5030 tablet_bootstrap.cc:492] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: Bootstrap starting.
I20260812 07:10:25.764391 4847 tablet_service.cc:1467] Processing CreateTablet for tablet 35c443fa283c49008268ff9fba1f05a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77ee9dbd3d0e4affa3d13e888fa597d2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260812 07:10:25.764750 5030 tablet_bootstrap.cc:654] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.764950 4847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 35c443fa283c49008268ff9fba1f05a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:25.766897 5030 tablet_bootstrap.cc:492] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: No bootstrap required, opened a new log
I20260812 07:10:25.767064 5030 ts_tablet_manager.cc:1397] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260812 07:10:25.767851 5030 raft_consensus.cc:348] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.768209 5030 raft_consensus.cc:374] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.768316 5030 raft_consensus.cc:729] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce88a0bdedb4489b8499b2b44abe68d, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.768563 5030 consensus_queue.cc:260] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.769143 5030 ts_tablet_manager.cc:1428] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:10:25.768990 5054 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.769392 5054 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:25.770187 5031 tablet_bootstrap.cc:492] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: Bootstrap starting.
I20260812 07:10:25.771859 5031 tablet_bootstrap.cc:654] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.774351 5032 tablet_bootstrap.cc:492] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: Bootstrap starting.
I20260812 07:10:25.776029 5032 tablet_bootstrap.cc:654] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:25.778613 5032 tablet_bootstrap.cc:492] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: No bootstrap required, opened a new log
I20260812 07:10:25.778852 5032 ts_tablet_manager.cc:1397] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260812 07:10:25.779595 5032 raft_consensus.cc:348] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.779908 5032 raft_consensus.cc:374] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.780045 5032 raft_consensus.cc:729] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a108ecfef49a4ced975671472692196b, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.780371 5032 consensus_queue.cc:260] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.780867 5044 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.781158 5044 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:25.781718 5032 ts_tablet_manager.cc:1428] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260812 07:10:25.782346 5031 tablet_bootstrap.cc:492] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: No bootstrap required, opened a new log
I20260812 07:10:25.782519 5031 ts_tablet_manager.cc:1397] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.007s
I20260812 07:10:25.783142 5031 raft_consensus.cc:348] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.783399 5031 raft_consensus.cc:374] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:25.783515 5031 raft_consensus.cc:729] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b002ca2e965a45f58223bbeef4d192d0, State: Initialized, Role: FOLLOWER
I20260812 07:10:25.783816 5031 consensus_queue.cc:260] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.784310 5049 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.784673 5049 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:25.785100 5031 ts_tablet_manager.cc:1428] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:10:25.867092 5049 raft_consensus.cc:482] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:25.867307 5049 raft_consensus.cc:504] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.868878 5049 leader_election.cc:288] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), a108ecfef49a4ced975671472692196b (127.21.181.67:39191)
I20260812 07:10:25.888720 5026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.869514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:56047 (local address 127.21.181.65:43827)
0812 07:10:25.869732 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.869738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.880880 (+ 11142us) server_negotiation.cc:407] Connection header received
0812 07:10:25.881351 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.881357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.881437 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.881503 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.882480 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.883621 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.886119 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.887438 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.887586 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.887659 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.888239 (+ 580us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.888400 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:25.889302 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.869915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:54669 (local address 127.21.181.67:39191)
0812 07:10:25.870095 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.870101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.882265 (+ 12164us) server_negotiation.cc:407] Connection header received
0812 07:10:25.882709 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.882714 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.882789 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.882848 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:25.883640 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.884780 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.887080 (+ 2300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.888228 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.888337 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.888402 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.888932 (+ 530us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.889051 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:25.890136 5120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.869405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:39191 (local address 127.21.181.66:54669)
0812 07:10:25.882160 (+ 12755us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.882179 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.882548 (+ 369us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.883040 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.883045 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.883069 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.883413 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.883424 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.884950 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.884955 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.886900 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.886916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.888592 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.888597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.888749 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.888780 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.889654 (+ 874us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.889782 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12592,"thread_start_us":12868,"threads_started":1}
I20260812 07:10:25.890801 5119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.869062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43827 (local address 127.21.181.66:56047)
0812 07:10:25.880766 (+ 11704us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.880788 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.881170 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.881739 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.881745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.881776 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:25.882256 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.882273 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.883790 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.883795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.885933 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.885949 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.887850 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.887856 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.888029 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.888063 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.890352 (+ 2289us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.890483 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11561,"thread_start_us":11843,"threads_started":1}
I20260812 07:10:25.891455 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c443fa283c49008268ff9fba1f05a7" candidate_uuid: "b002ca2e965a45f58223bbeef4d192d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a108ecfef49a4ced975671472692196b" is_pre_election: true
I20260812 07:10:25.891446 4328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c443fa283c49008268ff9fba1f05a7" candidate_uuid: "b002ca2e965a45f58223bbeef4d192d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d" is_pre_election: true
I20260812 07:10:25.891703 4892 raft_consensus.cc:2454] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b002ca2e965a45f58223bbeef4d192d0 in term 0.
I20260812 07:10:25.891733 4328 raft_consensus.cc:2454] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b002ca2e965a45f58223bbeef4d192d0 in term 0.
I20260812 07:10:25.892354 4466 leader_election.cc:302] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, b002ca2e965a45f58223bbeef4d192d0; no voters:
I20260812 07:10:25.892812 5049 raft_consensus.cc:2781] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:25.892933 5049 raft_consensus.cc:482] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:25.893033 5049 raft_consensus.cc:3037] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.895375 5049 raft_consensus.cc:504] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.895915 5049 leader_election.cc:288] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [CANDIDATE]: Term 1 election: Requested vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), a108ecfef49a4ced975671472692196b (127.21.181.67:39191)
I20260812 07:10:25.896428 4328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c443fa283c49008268ff9fba1f05a7" candidate_uuid: "b002ca2e965a45f58223bbeef4d192d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d"
I20260812 07:10:25.896673 4328 raft_consensus.cc:3037] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.896621 4892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "35c443fa283c49008268ff9fba1f05a7" candidate_uuid: "b002ca2e965a45f58223bbeef4d192d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a108ecfef49a4ced975671472692196b"
I20260812 07:10:25.896835 4892 raft_consensus.cc:3037] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:25.899017 4892 raft_consensus.cc:2454] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b002ca2e965a45f58223bbeef4d192d0 in term 1.
I20260812 07:10:25.899493 4466 leader_election.cc:302] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, b002ca2e965a45f58223bbeef4d192d0; no voters:
I20260812 07:10:25.899820 5049 raft_consensus.cc:2781] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:25.900107 4328 raft_consensus.cc:2454] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b002ca2e965a45f58223bbeef4d192d0 in term 1.
I20260812 07:10:25.901002 5049 raft_consensus.cc:686] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [term 1 LEADER]: Becoming Leader. State: Replica: b002ca2e965a45f58223bbeef4d192d0, State: Running, Role: LEADER
I20260812 07:10:25.901537 5049 consensus_queue.cc:237] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:25.904105 5121 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: TxnStatusTablet state changed. Reason: New leader b002ca2e965a45f58223bbeef4d192d0. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.904465 5121 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:10:25.905603 4043 catalog_manager.cc:5495] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } health_report { overall_health: UNKNOWN } } }
I20260812 07:10:25.910037 5123 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260812 07:10:25.910277 5123 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260812 07:10:25.945469 5110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.937770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36257 (local address 127.0.0.1:57174)
0812 07:10:25.937984 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:25.938004 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:25.938153 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:25.938948 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:25.938952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:25.938966 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:10:25.939160 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.939172 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.940729 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.940735 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.942291 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:25.942308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.943451 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.943457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:25.943644 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.943721 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:10:25.944912 (+ 1191us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:10:25.944936 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:10:25.945018 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 07:10:25.945169 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:10:25.946346 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.937409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.21.181.66:36257)
0812 07:10:25.937612 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.937618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:25.938329 (+ 711us) server_negotiation.cc:407] Connection header received
0812 07:10:25.938432 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:25.938437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:25.938608 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:25.938751 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:25.939340 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.940554 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.942491 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:25.943151 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:25.943305 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:25.943951 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:25.944706 (+ 755us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:25.944850 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:25.945954 (+ 1104us) server_negotiation.cc:299] Negotiation successful
0812 07:10:25.946056 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:10:25.951779 4892 raft_consensus.cc:1261] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Refusing update from remote peer b002ca2e965a45f58223bbeef4d192d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:25.952001 4328 raft_consensus.cc:1261] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Refusing update from remote peer b002ca2e965a45f58223bbeef4d192d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:25.952674 5049 consensus_queue.cc:1035] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:25.952988 5121 consensus_queue.cc:1035] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:25.968616 5044 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: TxnStatusTablet state changed. Reason: New leader b002ca2e965a45f58223bbeef4d192d0. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.968972 5044 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:25.972812 5061 mvcc.cc:205] Tried to move back new op lower bound from 7317580291888451584 to 7317580291697377280. Current Snapshot: MvccSnapshot[applied={T|T < 7317580291888451584}]
I20260812 07:10:25.971967 5054 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: TxnStatusTablet state changed. Reason: New leader b002ca2e965a45f58223bbeef4d192d0. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.978363 5121 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.978638 5121 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:10:25.982179 5121 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.982530 5054 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:25.983355 5054 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.983639 5121 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P b002ca2e965a45f58223bbeef4d192d0: This TxnStatusTablet replica's current role is: LEADER
I20260812 07:10:25.983901 5054 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P dce88a0bdedb4489b8499b2b44abe68d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:25.984652 5044 tablet_replica.cc:439] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b002ca2e965a45f58223bbeef4d192d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } } }
I20260812 07:10:25.984942 5044 tablet_replica.cc:442] T 35c443fa283c49008268ff9fba1f05a7 P a108ecfef49a4ced975671472692196b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260812 07:10:26.014555 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:25.999225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57820 (local address 127.21.181.67:39191)
0812 07:10:25.999405 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:25.999412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.007099 (+ 7687us) server_negotiation.cc:407] Connection header received
0812 07:10:26.007352 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.007358 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.007485 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.007645 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:26.008414 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.009532 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.011603 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.012291 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.012392 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.013050 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:26.013795 (+ 745us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:26.013925 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.014166 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.014254 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:10:26.058404 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.044505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:32867 (local address 127.21.181.66:36257)
0812 07:10:26.044690 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.044697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.051495 (+ 6798us) server_negotiation.cc:407] Connection header received
0812 07:10:26.051945 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.051951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.052040 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.052130 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:26.052848 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.053959 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.056037 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.056784 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.056880 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.057570 (+ 690us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:26.057706 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:26.057768 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.057987 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.058072 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:26.059171 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.044827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36257 (local address 127.21.181.67:32867)
0812 07:10:26.051381 (+ 6554us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:26.051403 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:26.051767 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:26.052335 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:26.052340 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:26.052353 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:10:26.052673 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:26.052684 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.054153 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.054158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:26.055866 (+ 1708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:26.055882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.057080 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.057087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:26.057262 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.057353 (+ 91us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:10:26.057850 (+ 497us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:10:26.057861 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:10:26.058662 (+ 801us) client_negotiation.cc:240] Negotiation successful
0812 07:10:26.058804 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6367,"thread_start_us":6686,"threads_started":1}
I20260812 07:10:26.068025 5044 consensus_queue.cc:1035] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:26.070765 4327 raft_consensus.cc:1261] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Refusing update from remote peer a108ecfef49a4ced975671472692196b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:26.071355 5044 consensus_queue.cc:1035] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:26.119797 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.112348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:36849 (local address 127.21.181.67:39191)
0812 07:10:26.112663 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.112671 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.114463 (+ 1792us) server_negotiation.cc:407] Connection header received
0812 07:10:26.114638 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.114643 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.114712 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.114780 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:26.115327 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.116328 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.118112 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.118552 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.118673 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.119091 (+ 418us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:26.119193 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:26.119269 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.119427 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.119502 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174}
I20260812 07:10:26.120024 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.111930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:39191 (local address 127.21.181.66:36849)
0812 07:10:26.114362 (+ 2432us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:26.114377 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:26.114519 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:26.114819 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:26.114824 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:26.114836 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0812 07:10:26.115178 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:26.115187 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.116473 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.116478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:26.117976 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:26.117990 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.118735 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.118745 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:26.118876 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.118930 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0812 07:10:26.119286 (+ 356us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0812 07:10:26.119297 (+ 11us) client_negotiation.cc:769] Sending connection context
0812 07:10:26.119380 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:10:26.119516 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2239,"thread_start_us":2077,"threads_started":1}
I20260812 07:10:26.170570 4043 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:26.170792 4043 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:26.173894 4043 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Sending DeleteTablet for 3 replicas of tablet c80baf852af149b2bf64c5cebf32095c
I20260812 07:10:26.174921 4290 tablet_service.cc:1514] Processing DeleteTablet for tablet c80baf852af149b2bf64c5cebf32095c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56656
I20260812 07:10:26.174984 4566 tablet_service.cc:1514] Processing DeleteTablet for tablet c80baf852af149b2bf64c5cebf32095c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57162
I20260812 07:10:26.175333 4847 tablet_service.cc:1514] Processing DeleteTablet for tablet c80baf852af149b2bf64c5cebf32095c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57818
I20260812 07:10:26.213074 5159 tablet_replica.cc:330] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: stopping tablet replica
I20260812 07:10:26.213424 5159 raft_consensus.cc:2229] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:26.214185 5159 raft_consensus.cc:2258] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.215612 5159 ts_tablet_manager.cc:1905] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.220579 5153 tablet_replica.cc:330] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: stopping tablet replica
I20260812 07:10:26.220907 5153 raft_consensus.cc:2229] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:26.223958 5153 raft_consensus.cc:2258] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.226118 5153 ts_tablet_manager.cc:1905] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.226666 5157 tablet_replica.cc:330] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: stopping tablet replica
I20260812 07:10:26.232282 5157 raft_consensus.cc:2229] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:26.232666 5157 raft_consensus.cc:2258] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.233903 5157 ts_tablet_manager.cc:1905] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.239396 5153 ts_tablet_manager.cc:1918] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:10:26.239530 5153 log.cc:1196] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/c80baf852af149b2bf64c5cebf32095c
I20260812 07:10:26.239998 5153 ts_tablet_manager.cc:1939] T c80baf852af149b2bf64c5cebf32095c P dce88a0bdedb4489b8499b2b44abe68d: Deleting consensus metadata
I20260812 07:10:26.241281 4016 catalog_manager.cc:4841] TS dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827): tablet c80baf852af149b2bf64c5cebf32095c (table test-table [id=7465cc5204474160842e39bbe72bb3dc]) successfully deleted
I20260812 07:10:26.244134 5159 ts_tablet_manager.cc:1918] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:10:26.244305 5159 log.cc:1196] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/c80baf852af149b2bf64c5cebf32095c
I20260812 07:10:26.244777 5159 ts_tablet_manager.cc:1939] T c80baf852af149b2bf64c5cebf32095c P a108ecfef49a4ced975671472692196b: Deleting consensus metadata
I20260812 07:10:26.245890 4014 catalog_manager.cc:4841] TS a108ecfef49a4ced975671472692196b (127.21.181.67:39191): tablet c80baf852af149b2bf64c5cebf32095c (table test-table [id=7465cc5204474160842e39bbe72bb3dc]) successfully deleted
I20260812 07:10:26.248479 5157 ts_tablet_manager.cc:1918] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:10:26.248667 5157 log.cc:1196] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/c80baf852af149b2bf64c5cebf32095c
I20260812 07:10:26.249136 5157 ts_tablet_manager.cc:1939] T c80baf852af149b2bf64c5cebf32095c P b002ca2e965a45f58223bbeef4d192d0: Deleting consensus metadata
I20260812 07:10:26.250209 4016 catalog_manager.cc:4841] TS b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257): tablet c80baf852af149b2bf64c5cebf32095c (table test-table [id=7465cc5204474160842e39bbe72bb3dc]) successfully deleted
Aug 12 07:10:26 dist-test-slave-lgr5 krb5kdc[3970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260812 07:10:26.292709 22229 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.111s user 0.001s sys 0.020s
Aug 12 07:10:26 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:26 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:10:26.332556 5177 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:26.300110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.21.181.126:43791)
0812 07:10:26.312457 (+ 12347us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.312463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.312775 (+ 312us) server_negotiation.cc:407] Connection header received
0812 07:10:26.312973 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.312979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.313053 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.313148 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:26.314448 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.315179 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.316592 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.317028 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.317139 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.332181 (+ 15042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48222: BlockingRecv error: recv got EOF from 127.0.0.1:48222 (error 108)
Metrics: {"server-negotiator.queue_time_us":12185,"thread_start_us":12073,"threads_started":1}
Aug 12 07:10:26 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM
I20260812 07:10:26.347003 5177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.333801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.21.181.126:43791)
0812 07:10:26.334003 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.334010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.334458 (+ 448us) server_negotiation.cc:407] Connection header received
0812 07:10:26.334622 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.334628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.334697 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.334779 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:26.335977 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.336754 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.337978 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.338401 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.338512 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.342064 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:26.342070 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:26.342073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:26.342107 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:26.344582 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:26.345177 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:26.345181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:26.345183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:26.345309 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:26.345668 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:26.345672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:26.345674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:26.345939 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:26.346178 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.346457 (+ 279us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.346668 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:10:26.350270 4043 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:26.350891 4043 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:26.357765 4290 tablet_service.cc:1467] Processing CreateTablet for tablet bfbe6c81e8954dfca2e78110919a450a (DEFAULT_TABLE table=test-table [id=90880fc277034401b16b374d9601aaa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:26.358246 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfbe6c81e8954dfca2e78110919a450a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:26.358192 4847 tablet_service.cc:1467] Processing CreateTablet for tablet bfbe6c81e8954dfca2e78110919a450a (DEFAULT_TABLE table=test-table [id=90880fc277034401b16b374d9601aaa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:26.358605 4847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfbe6c81e8954dfca2e78110919a450a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:26.365053 4566 tablet_service.cc:1467] Processing CreateTablet for tablet bfbe6c81e8954dfca2e78110919a450a (DEFAULT_TABLE table=test-table [id=90880fc277034401b16b374d9601aaa8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:26.365614 4566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bfbe6c81e8954dfca2e78110919a450a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:26.377133 5187 tablet_bootstrap.cc:492] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Bootstrap starting.
I20260812 07:10:26.378445 5191 tablet_bootstrap.cc:492] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Bootstrap starting.
I20260812 07:10:26.380636 5190 tablet_bootstrap.cc:492] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Bootstrap starting.
I20260812 07:10:26.381946 5191 tablet_bootstrap.cc:654] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:26.383185 5187 tablet_bootstrap.cc:654] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:26.384205 5190 tablet_bootstrap.cc:654] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:26.392001 5190 tablet_bootstrap.cc:492] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: No bootstrap required, opened a new log
I20260812 07:10:26.394716 5191 tablet_bootstrap.cc:492] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: No bootstrap required, opened a new log
I20260812 07:10:26.395033 5191 ts_tablet_manager.cc:1397] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.006s
I20260812 07:10:26.395421 5187 tablet_bootstrap.cc:492] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: No bootstrap required, opened a new log
I20260812 07:10:26.395592 5187 ts_tablet_manager.cc:1397] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.003s
I20260812 07:10:26.396384 5191 raft_consensus.cc:348] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.396454 5187 raft_consensus.cc:348] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.396708 5187 raft_consensus.cc:374] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:26.396826 5187 raft_consensus.cc:729] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce88a0bdedb4489b8499b2b44abe68d, State: Initialized, Role: FOLLOWER
I20260812 07:10:26.396831 5191 raft_consensus.cc:374] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:26.397044 5191 raft_consensus.cc:729] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b002ca2e965a45f58223bbeef4d192d0, State: Initialized, Role: FOLLOWER
I20260812 07:10:26.397130 5187 consensus_queue.cc:260] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.397511 5191 consensus_queue.cc:260] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.398550 5191 ts_tablet_manager.cc:1428] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260812 07:10:26.401962 5187 ts_tablet_manager.cc:1428] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260812 07:10:26.404249 5190 ts_tablet_manager.cc:1397] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.006s
I20260812 07:10:26.405138 5190 raft_consensus.cc:348] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.405401 5190 raft_consensus.cc:374] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:26.405507 5190 raft_consensus.cc:729] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a108ecfef49a4ced975671472692196b, State: Initialized, Role: FOLLOWER
I20260812 07:10:26.405752 5190 consensus_queue.cc:260] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.406327 5190 ts_tablet_manager.cc:1428] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260812 07:10:26.436124 5044 raft_consensus.cc:482] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:26.436360 5044 raft_consensus.cc:504] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.437216 5044 leader_election.cc:288] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:26.437747 4327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbe6c81e8954dfca2e78110919a450a" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d" is_pre_election: true
I20260812 07:10:26.438043 4327 raft_consensus.cc:2454] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 0.
I20260812 07:10:26.438066 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbe6c81e8954dfca2e78110919a450a" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0" is_pre_election: true
I20260812 07:10:26.438354 4625 raft_consensus.cc:2454] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 0.
I20260812 07:10:26.438545 4760 leader_election.cc:302] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:26.439080 5044 raft_consensus.cc:2781] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:26.439193 5044 raft_consensus.cc:482] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:26.439317 5044 raft_consensus.cc:3037] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:26.442607 5044 raft_consensus.cc:504] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.443295 5044 leader_election.cc:288] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 election: Requested vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:26.443848 4327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbe6c81e8954dfca2e78110919a450a" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d"
I20260812 07:10:26.444170 4327 raft_consensus.cc:3037] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:26.444205 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bfbe6c81e8954dfca2e78110919a450a" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0"
I20260812 07:10:26.444427 4625 raft_consensus.cc:3037] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:26.447116 4327 raft_consensus.cc:2454] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 1.
I20260812 07:10:26.447494 4625 raft_consensus.cc:2454] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 1.
I20260812 07:10:26.447640 4760 leader_election.cc:302] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:26.448045 5044 raft_consensus.cc:2781] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:26.448269 5044 raft_consensus.cc:686] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 1 LEADER]: Becoming Leader. State: Replica: a108ecfef49a4ced975671472692196b, State: Running, Role: LEADER
I20260812 07:10:26.448539 5044 consensus_queue.cc:237] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.451975 4043 catalog_manager.cc:5495] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b reported cstate change: term changed from 0 to 1, leader changed from <none> to a108ecfef49a4ced975671472692196b (127.21.181.67). New cstate: current_term: 1 leader_uuid: "a108ecfef49a4ced975671472692196b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } health_report { overall_health: HEALTHY } } }
I20260812 07:10:26.482266 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.475398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57836 (local address 127.21.181.67:39191)
0812 07:10:26.475585 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.475591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.475740 (+ 149us) server_negotiation.cc:407] Connection header received
0812 07:10:26.475942 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.475948 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.476040 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.476165 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:26.476904 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.478090 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.480125 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.480811 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.480978 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.481508 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:26.481656 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:26.481747 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.481891 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.481976 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:10:26.484987 4625 raft_consensus.cc:1261] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Refusing update from remote peer a108ecfef49a4ced975671472692196b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:26.486270 4327 raft_consensus.cc:1261] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Refusing update from remote peer a108ecfef49a4ced975671472692196b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:26.485741 5044 consensus_queue.cc:1035] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:26.486804 5148 consensus_queue.cc:1035] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:26.500073 5063 mvcc.cc:205] Tried to move back new op lower bound from 7317580294076841984 to 7317580293937094656. Current Snapshot: MvccSnapshot[applied={T|T < 7317580294076841984}]
I20260812 07:10:26.508813 4048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:26.509092 4048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:26.512483 4048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Sending DeleteTablet for 3 replicas of tablet bfbe6c81e8954dfca2e78110919a450a
I20260812 07:10:26.513249 4847 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbe6c81e8954dfca2e78110919a450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57818
I20260812 07:10:26.513236 4290 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbe6c81e8954dfca2e78110919a450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56656
I20260812 07:10:26.513509 5153 tablet_replica.cc:330] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: stopping tablet replica
I20260812 07:10:26.513514 5159 tablet_replica.cc:330] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: stopping tablet replica
I20260812 07:10:26.513572 4566 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbe6c81e8954dfca2e78110919a450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57162
I20260812 07:10:26.513695 5153 raft_consensus.cc:2229] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:26.513712 5159 raft_consensus.cc:2229] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:26.513849 4565 tablet_service.cc:1514] Processing DeleteTablet for tablet bfbe6c81e8954dfca2e78110919a450a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57162
I20260812 07:10:26.513901 5153 raft_consensus.cc:2258] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.514009 5159 raft_consensus.cc:2258] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.514186 5157 tablet_replica.cc:330] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: stopping tablet replica
I20260812 07:10:26.514381 5157 raft_consensus.cc:2229] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:26.514588 5157 raft_consensus.cc:2258] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.514906 5159 ts_tablet_manager.cc:1905] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.515442 5153 ts_tablet_manager.cc:1905] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.515940 4048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:26.516563 4048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:26.522053 5153 ts_tablet_manager.cc:1918] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:26.522203 5153 log.cc:1196] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/bfbe6c81e8954dfca2e78110919a450a
I20260812 07:10:26.522650 5153 ts_tablet_manager.cc:1939] T bfbe6c81e8954dfca2e78110919a450a P dce88a0bdedb4489b8499b2b44abe68d: Deleting consensus metadata
I20260812 07:10:26.523689 4847 tablet_service.cc:1467] Processing CreateTablet for tablet 3d52b3ada5b0476fa483747a8e273b49 (DEFAULT_TABLE table=test-table [id=a7196b1ca24b4f7db4b028e025dfefa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:26.524595 5157 ts_tablet_manager.cc:1905] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.524613 4016 catalog_manager.cc:4841] TS dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827): tablet bfbe6c81e8954dfca2e78110919a450a (table test-table [id=90880fc277034401b16b374d9601aaa8]) successfully deleted
I20260812 07:10:26.525241 4290 tablet_service.cc:1467] Processing CreateTablet for tablet 3d52b3ada5b0476fa483747a8e273b49 (DEFAULT_TABLE table=test-table [id=a7196b1ca24b4f7db4b028e025dfefa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:26.525636 4290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d52b3ada5b0476fa483747a8e273b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:26.525892 5159 ts_tablet_manager.cc:1918] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:26.526028 5159 log.cc:1196] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/bfbe6c81e8954dfca2e78110919a450a
I20260812 07:10:26.526494 5159 ts_tablet_manager.cc:1939] T bfbe6c81e8954dfca2e78110919a450a P a108ecfef49a4ced975671472692196b: Deleting consensus metadata
I20260812 07:10:26.527915 4847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d52b3ada5b0476fa483747a8e273b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:26.528837 4565 tablet_service.cc:1467] Processing CreateTablet for tablet 3d52b3ada5b0476fa483747a8e273b49 (DEFAULT_TABLE table=test-table [id=a7196b1ca24b4f7db4b028e025dfefa7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:26.529244 4565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d52b3ada5b0476fa483747a8e273b49. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:26.532240 4014 catalog_manager.cc:4841] TS a108ecfef49a4ced975671472692196b (127.21.181.67:39191): tablet bfbe6c81e8954dfca2e78110919a450a (table test-table [id=90880fc277034401b16b374d9601aaa8]) successfully deleted
I20260812 07:10:26.534032 5157 ts_tablet_manager.cc:1918] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:26.534163 5157 log.cc:1196] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/bfbe6c81e8954dfca2e78110919a450a
I20260812 07:10:26.534397 5190 tablet_bootstrap.cc:492] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Bootstrap starting.
I20260812 07:10:26.534562 5157 ts_tablet_manager.cc:1939] T bfbe6c81e8954dfca2e78110919a450a P b002ca2e965a45f58223bbeef4d192d0: Deleting consensus metadata
I20260812 07:10:26.535482 4016 catalog_manager.cc:4841] TS b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257): tablet bfbe6c81e8954dfca2e78110919a450a (table test-table [id=90880fc277034401b16b374d9601aaa8]) successfully deleted
W20260812 07:10:26.535873 4016 catalog_manager.cc:4808] TS b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257): delete failed for tablet bfbe6c81e8954dfca2e78110919a450a because the tablet was not found. No further retry: Not found: Tablet not found: bfbe6c81e8954dfca2e78110919a450a
I20260812 07:10:26.535954 5190 tablet_bootstrap.cc:654] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:26.539314 5187 tablet_bootstrap.cc:492] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Bootstrap starting.
I20260812 07:10:26.539752 5191 tablet_bootstrap.cc:492] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Bootstrap starting.
I20260812 07:10:26.541409 5191 tablet_bootstrap.cc:654] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:26.541936 5187 tablet_bootstrap.cc:654] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:26.550683 5190 tablet_bootstrap.cc:492] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: No bootstrap required, opened a new log
I20260812 07:10:26.550887 5190 ts_tablet_manager.cc:1397] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20260812 07:10:26.551741 5190 raft_consensus.cc:348] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.552033 5190 raft_consensus.cc:374] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:26.552134 5190 raft_consensus.cc:729] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a108ecfef49a4ced975671472692196b, State: Initialized, Role: FOLLOWER
I20260812 07:10:26.552418 5190 consensus_queue.cc:260] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.553067 5190 ts_tablet_manager.cc:1428] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260812 07:10:26.553274 5191 tablet_bootstrap.cc:492] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: No bootstrap required, opened a new log
I20260812 07:10:26.553457 5191 ts_tablet_manager.cc:1397] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20260812 07:10:26.554148 5191 raft_consensus.cc:348] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.554416 5191 raft_consensus.cc:374] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:26.554514 5191 raft_consensus.cc:729] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b002ca2e965a45f58223bbeef4d192d0, State: Initialized, Role: FOLLOWER
I20260812 07:10:26.554772 5191 consensus_queue.cc:260] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.556636 5191 ts_tablet_manager.cc:1428] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260812 07:10:26.556952 5187 tablet_bootstrap.cc:492] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: No bootstrap required, opened a new log
I20260812 07:10:26.557159 5187 ts_tablet_manager.cc:1397] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.006s
I20260812 07:10:26.558043 5187 raft_consensus.cc:348] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.558316 5187 raft_consensus.cc:374] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:26.558406 5187 raft_consensus.cc:729] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dce88a0bdedb4489b8499b2b44abe68d, State: Initialized, Role: FOLLOWER
I20260812 07:10:26.558697 5187 consensus_queue.cc:260] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.559278 5187 ts_tablet_manager.cc:1428] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260812 07:10:26.570621 5044 raft_consensus.cc:482] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:26.570815 5044 raft_consensus.cc:504] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.571434 5044 leader_election.cc:288] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:26.571993 4328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d52b3ada5b0476fa483747a8e273b49" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d" is_pre_election: true
I20260812 07:10:26.572028 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d52b3ada5b0476fa483747a8e273b49" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0" is_pre_election: true
I20260812 07:10:26.572299 4328 raft_consensus.cc:2454] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 0.
I20260812 07:10:26.572309 4625 raft_consensus.cc:2454] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 0.
I20260812 07:10:26.572794 4760 leader_election.cc:302] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:26.573109 5044 raft_consensus.cc:2781] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:26.573222 5044 raft_consensus.cc:482] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:26.573308 5044 raft_consensus.cc:3037] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:26.576498 5044 raft_consensus.cc:504] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.577183 5044 leader_election.cc:288] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 election: Requested vote from peers dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827), b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257)
I20260812 07:10:26.577618 4328 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d52b3ada5b0476fa483747a8e273b49" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dce88a0bdedb4489b8499b2b44abe68d"
I20260812 07:10:26.577864 4328 raft_consensus.cc:3037] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:26.577836 4625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d52b3ada5b0476fa483747a8e273b49" candidate_uuid: "a108ecfef49a4ced975671472692196b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b002ca2e965a45f58223bbeef4d192d0"
I20260812 07:10:26.578032 4625 raft_consensus.cc:3037] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:26.580677 4328 raft_consensus.cc:2454] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 1.
I20260812 07:10:26.581067 4625 raft_consensus.cc:2454] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a108ecfef49a4ced975671472692196b in term 1.
I20260812 07:10:26.581079 4760 leader_election.cc:302] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a108ecfef49a4ced975671472692196b, dce88a0bdedb4489b8499b2b44abe68d; no voters:
I20260812 07:10:26.581338 5044 raft_consensus.cc:2781] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:26.581568 5044 raft_consensus.cc:686] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 1 LEADER]: Becoming Leader. State: Replica: a108ecfef49a4ced975671472692196b, State: Running, Role: LEADER
I20260812 07:10:26.581830 5044 consensus_queue.cc:237] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } }
I20260812 07:10:26.584252 4048 catalog_manager.cc:5495] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b reported cstate change: term changed from 0 to 1, leader changed from <none> to a108ecfef49a4ced975671472692196b (127.21.181.67). New cstate: current_term: 1 leader_uuid: "a108ecfef49a4ced975671472692196b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a108ecfef49a4ced975671472692196b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 39191 } health_report { overall_health: HEALTHY } } }
I20260812 07:10:26.614184 4625 raft_consensus.cc:1261] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Refusing update from remote peer a108ecfef49a4ced975671472692196b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:26.614801 4328 raft_consensus.cc:1261] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Refusing update from remote peer a108ecfef49a4ced975671472692196b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:26.614749 5148 consensus_queue.cc:1035] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b002ca2e965a45f58223bbeef4d192d0" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:26.615304 5148 consensus_queue.cc:1035] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dce88a0bdedb4489b8499b2b44abe68d" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:26.677798 4048 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:26.678102 4048 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:26.681253 4048 catalog_manager.cc:5782] T 00000000000000000000000000000000 P aced4c0700214bf6a5ff3870b8debb38: Sending DeleteTablet for 3 replicas of tablet 3d52b3ada5b0476fa483747a8e273b49
I20260812 07:10:26.682068 4290 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d52b3ada5b0476fa483747a8e273b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56656
I20260812 07:10:26.682384 5153 tablet_replica.cc:330] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: stopping tablet replica
I20260812 07:10:26.682580 5153 raft_consensus.cc:2229] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:26.682565 4847 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d52b3ada5b0476fa483747a8e273b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57818
I20260812 07:10:26.682643 4565 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d52b3ada5b0476fa483747a8e273b49 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:26 UTC) from {username='oryx', principal='oryx/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:57162
I20260812 07:10:26.682792 5153 raft_consensus.cc:2258] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.682924 5157 tablet_replica.cc:330] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: stopping tablet replica
I20260812 07:10:26.683082 5159 tablet_replica.cc:330] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: stopping tablet replica
I20260812 07:10:26.683109 5157 raft_consensus.cc:2229] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:26.683238 5159 raft_consensus.cc:2229] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:26.683314 5157 raft_consensus.cc:2258] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.683571 5159 raft_consensus.cc:2258] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:26.684054 5153 ts_tablet_manager.cc:1905] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.684883 5159 ts_tablet_manager.cc:1905] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.684894 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 4174
I20260812 07:10:26.690732 5153 ts_tablet_manager.cc:1918] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:10:26.690874 5153 log.cc:1196] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/3d52b3ada5b0476fa483747a8e273b49
I20260812 07:10:26.691291 5153 ts_tablet_manager.cc:1939] T 3d52b3ada5b0476fa483747a8e273b49 P dce88a0bdedb4489b8499b2b44abe68d: Deleting consensus metadata
I20260812 07:10:26.692792 5157 ts_tablet_manager.cc:1905] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:26.697607 5157 ts_tablet_manager.cc:1918] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:10:26.697731 5157 log.cc:1196] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/3d52b3ada5b0476fa483747a8e273b49
I20260812 07:10:26.698109 5157 ts_tablet_manager.cc:1939] T 3d52b3ada5b0476fa483747a8e273b49 P b002ca2e965a45f58223bbeef4d192d0: Deleting consensus metadata
I20260812 07:10:26.699149 4016 catalog_manager.cc:4841] TS b002ca2e965a45f58223bbeef4d192d0 (127.21.181.66:36257): tablet 3d52b3ada5b0476fa483747a8e273b49 (table test-table [id=a7196b1ca24b4f7db4b028e025dfefa7]) successfully deleted
I20260812 07:10:26.700605 4016 catalog_manager.cc:4841] TS dce88a0bdedb4489b8499b2b44abe68d (127.21.181.65:43827): tablet 3d52b3ada5b0476fa483747a8e273b49 (table test-table [id=a7196b1ca24b4f7db4b028e025dfefa7]) successfully deleted
I20260812 07:10:26.701895 5159 ts_tablet_manager.cc:1918] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260812 07:10:26.702049 5159 log.cc:1196] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/3d52b3ada5b0476fa483747a8e273b49
I20260812 07:10:26.702510 5159 ts_tablet_manager.cc:1939] T 3d52b3ada5b0476fa483747a8e273b49 P a108ecfef49a4ced975671472692196b: Deleting consensus metadata
I20260812 07:10:26.703756 4014 catalog_manager.cc:4841] TS a108ecfef49a4ced975671472692196b (127.21.181.67:39191): tablet 3d52b3ada5b0476fa483747a8e273b49 (table test-table [id=a7196b1ca24b4f7db4b028e025dfefa7]) successfully deleted
Aug 12 07:10:26 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.65@KRBTEST.COM
I20260812 07:10:26.715600 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.688810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56666 (local address 127.21.181.65:43827)
0812 07:10:26.700541 (+ 11731us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.700547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.700568 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:10:26.700627 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.700632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.700790 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.702688 (+ 1898us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:26.702859 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.703821 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.705135 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.705538 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.705645 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.709956 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:26.709971 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:26.709988 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:26.710020 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:26.712932 (+ 2912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:26.713723 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:26.713730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:26.713732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:26.713853 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:26.714237 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:26.714240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:26.714242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:26.714585 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:26.714805 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.715153 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.715311 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11587,"thread_start_us":11469,"threads_started":1}
I20260812 07:10:26.716372 4418 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:26.840272 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 4174
I20260812 07:10:26.871299 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 4431
Aug 12 07:10:26 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.66@KRBTEST.COM
I20260812 07:10:26.885174 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:26.871993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57184 (local address 127.21.181.66:36257)
0812 07:10:26.872475 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:26.872479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:26.872496 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:10:26.872554 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:26.872558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:26.872616 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:26.872710 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:26.873881 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.874657 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.875827 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:26.876357 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:26.876453 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:26.880277 (+ 3824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:26.880289 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:26.880304 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:26.880335 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:26.882936 (+ 2601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:26.883485 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:26.883492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:26.883496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:26.883620 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:26.883881 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:26.883884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:26.883885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:26.884284 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:26.884499 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:26.884732 (+ 233us) server_negotiation.cc:299] Negotiation successful
0812 07:10:26.884890 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":250,"threads_started":1}
I20260812 07:10:26.885799 4701 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:27.007063 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 4431
I20260812 07:10:27.036895 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 4728
Aug 12 07:10:27 dist-test-slave-lgr5 krb5kdc[3970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.21.181.67@KRBTEST.COM
I20260812 07:10:27.051734 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:27.037616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57852 (local address 127.21.181.67:39191)
0812 07:10:27.038085 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:27.038089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:27.038106 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:10:27.038162 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:27.038165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:27.038227 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:27.038317 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:27.039676 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:27.040400 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:27.041714 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:27.042136 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:27.042233 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:27.046579 (+ 4346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:27.046595 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:27.046613 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:27.046650 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:27.049401 (+ 2751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:27.050025 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:27.050032 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:27.050034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:27.050170 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:27.050454 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:27.050457 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:27.050459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:27.050816 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:27.051040 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:27.051233 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 07:10:27.051397 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":239,"threads_started":1}
I20260812 07:10:27.052410 4981 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:27.176237 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 4728
I20260812 07:10:27.200735 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 3990
I20260812 07:10:27.201912 4122 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:27.318560 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 3990
2026-08-12T07:10:27Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5612 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:27 dist-test-slave-lgr5 krb5kdc[5236](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:27 dist-test-slave-lgr5 krb5kdc[5236](info): set up 2 sockets
Aug 12 07:10:27 dist-test-slave-lgr5 krb5kdc[5236](info): commencing operation
krb5kdc: starting...
W20260812 07:10:28.579469 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:10:28 dist-test-slave-lgr5 krb5kdc[5236](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:10:28.802953 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33675
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:40745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:33675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:29.120321 5264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:29.120592 5264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:29.120728 5264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:29.127595 5264 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:29.127704 5264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:29.127781 5264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:29.127866 5264 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:29.127933 5264 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:29.137261 5264 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:40745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:33675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:29.138408 5264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:29.139690 5264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:29.150493 5274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:29.150563 5269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:29.150535 5271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:29.151528 5264 server_base.cc:1034] running on GCE node
I20260812 07:10:29.152285 5264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:29.153513 5264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:29.154716 5264 hybrid_clock.cc:648] HybridClock initialized: now 1786518629154672 us; error 34 us; skew 500 ppm
I20260812 07:10:29.155028 5264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-12T07:10:29Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1901 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:29 dist-test-slave-lgr5 krb5kdc[5292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:29 dist-test-slave-lgr5 krb5kdc[5292](info): set up 2 sockets
Aug 12 07:10:29 dist-test-slave-lgr5 krb5kdc[5292](info): commencing operation
krb5kdc: starting...
W20260812 07:10:30.478745 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.003s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:10:30 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:10:30.703418 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37881
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:31.019865 5319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:31.020164 5319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:31.020262 5319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:31.027136 5319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:31.027227 5319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:31.027279 5319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:31.027333 5319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:31.027375 5319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:31.036782 5319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:31.037935 5319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:31.039131 5319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:31.050002 5331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:31.049945 5333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:31.049944 5330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:31.050705 5319 server_base.cc:1034] running on GCE node
I20260812 07:10:31.051440 5319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:31.052793 5319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:31.053997 5319 hybrid_clock.cc:648] HybridClock initialized: now 1786518631053934 us; error 48 us; skew 500 ppm
I20260812 07:10:31.054311 5319 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:31 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518631, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:31.057224 5319 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:10:31.058525 5319 webserver.cc:460] Webserver started at http://127.21.181.126:39797/ using document root <none> and password file <none>
I20260812 07:10:31.059128 5319 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:31.059224 5319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:31.059484 5319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:31.062362 5319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "15f2b666c83a4762a3a2d39cd28afc58"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "9d206c0edfa78c74bbe7c0a0b3017110"
server_key_iv: "8df44d0dda764ffa07fa37e449b1d903"
server_key_version: "encryptionkey@0"
I20260812 07:10:31.063174 5319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "15f2b666c83a4762a3a2d39cd28afc58"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "9d206c0edfa78c74bbe7c0a0b3017110"
server_key_iv: "8df44d0dda764ffa07fa37e449b1d903"
server_key_version: "encryptionkey@0"
I20260812 07:10:31.068260 5319 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 07:10:31.071439 5354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:31.072748 5319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:31.072896 5319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "15f2b666c83a4762a3a2d39cd28afc58"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "9d206c0edfa78c74bbe7c0a0b3017110"
server_key_iv: "8df44d0dda764ffa07fa37e449b1d903"
server_key_version: "encryptionkey@0"
I20260812 07:10:31.073081 5319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:31.084308 5319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:31.088153 5319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:31.088402 5319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:31.104270 5319 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:37881
I20260812 07:10:31.104290 5455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:37881 every 8 connection(s)
I20260812 07:10:31.105594 5319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:31.109114 5458 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:31.112613 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 5319
I20260812 07:10:31.112816 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:31.113432 22229 external_mini_cluster.cc:1428] Setting key b70a4624f58da65e91cdea8a992b5b3a
I20260812 07:10:31.119553 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Bootstrap starting.
I20260812 07:10:31.122967 5458 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:31.124101 5458 log.cc:824] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:31.127159 5458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: No bootstrap required, opened a new log
Aug 12 07:10:31 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:31.137001 5464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:31.115223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52574 (local address 127.21.181.126:37881)
0812 07:10:31.115949 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:31.115959 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:31.115998 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:10:31.118166 (+ 2168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:31.118187 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:31.118550 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:31.118845 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:31.120623 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:31.121919 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:31.123571 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:31.124293 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:31.124439 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:31.128891 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:31.128901 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:31.128918 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:31.128963 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:31.132833 (+ 3870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:31.133442 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:31.133450 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:31.133457 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:31.133670 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:31.133999 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:31.134004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:31.134007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:31.134661 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:31.134962 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:31.135263 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 07:10:31.135697 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":334,"threads_started":1}
I20260812 07:10:31.137315 5458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:31.137694 5458 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:31.137785 5458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f2b666c83a4762a3a2d39cd28afc58, State: Initialized, Role: FOLLOWER
I20260812 07:10:31.138353 5458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:31.138602 5458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:31.138770 5458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:31.138939 5458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:31.141112 5458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:31.141584 5458 leader_election.cc:302] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15f2b666c83a4762a3a2d39cd28afc58; no voters:
I20260812 07:10:31.142108 5458 leader_election.cc:288] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:31.142213 5467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:31.142823 5467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 LEADER]: Becoming Leader. State: Replica: 15f2b666c83a4762a3a2d39cd28afc58, State: Running, Role: LEADER
I20260812 07:10:31.143360 5467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:31.143906 5458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:31.146286 5469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15f2b666c83a4762a3a2d39cd28afc58. Latest consensus state: current_term: 1 leader_uuid: "15f2b666c83a4762a3a2d39cd28afc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } } }
I20260812 07:10:31.146356 5468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15f2b666c83a4762a3a2d39cd28afc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } } }
I20260812 07:10:31.146756 5469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:31.146760 5468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:31.148145 5475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:31.153350 5475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:31.161139 5475 catalog_manager.cc:1347] Generated new cluster ID: c0b56e25862a4d8099356d5d397724c3
I20260812 07:10:31.161249 5475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:31.196044 5475 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:31.197818 5475 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:31.204736 5475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Generated new TSK 0
I20260812 07:10:31.205503 5475 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:10:31.265167 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37881
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:31.588500 5504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:31.588778 5504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:31.588917 5504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:31.595964 5504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:31.596127 5504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:31.596284 5504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:31.605623 5504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:31.606792 5504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:31.608047 5504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:31.620322 5521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:31.620370 5520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:31.620398 5524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:31.621344 5504 server_base.cc:1034] running on GCE node
I20260812 07:10:31.621886 5504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:31.622742 5504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:31.623929 5504 hybrid_clock.cc:648] HybridClock initialized: now 1786518631623888 us; error 31 us; skew 500 ppm
I20260812 07:10:31.624246 5504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:31 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518631, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:31.631485 5504 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:10:31.632907 5504 webserver.cc:460] Webserver started at http://127.21.181.65:43435/ using document root <none> and password file <none>
I20260812 07:10:31.633548 5504 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:31.633661 5504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:31.633950 5504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:31.636759 5504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "266ccc3a364c4c98a30ded3e1a0fb047"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "e17c8cec2e4f0cee5d9f590afda4135b"
server_key_iv: "66cb926bba828c5e1a4b47b57857d3ac"
server_key_version: "encryptionkey@0"
I20260812 07:10:31.637538 5504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "266ccc3a364c4c98a30ded3e1a0fb047"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "e17c8cec2e4f0cee5d9f590afda4135b"
server_key_iv: "66cb926bba828c5e1a4b47b57857d3ac"
server_key_version: "encryptionkey@0"
I20260812 07:10:31.642248 5504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:10:31.645547 5543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:31.646803 5504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260812 07:10:31.646946 5504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "266ccc3a364c4c98a30ded3e1a0fb047"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "e17c8cec2e4f0cee5d9f590afda4135b"
server_key_iv: "66cb926bba828c5e1a4b47b57857d3ac"
server_key_version: "encryptionkey@0"
I20260812 07:10:31.647132 5504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:31.662851 5504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:31.666451 5504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:31.666703 5504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:31.667564 5504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:31.668860 5504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:31.668936 5504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:31.669051 5504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:31.669106 5504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:31.699317 5504 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:42451
I20260812 07:10:31.699357 5748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:42451 every 8 connection(s)
I20260812 07:10:31.700528 5504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:31.701722 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 5504
I20260812 07:10:31.701948 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:31.702711 22229 external_mini_cluster.cc:1428] Setting key cb56a6c6046526c477b57320d78e3971
Aug 12 07:10:31 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518631, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:31.719383 5754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:31.703152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:40205 (local address 127.21.181.126:37881)
0812 07:10:31.703621 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:31.703627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:31.704592 (+ 965us) server_negotiation.cc:407] Connection header received
0812 07:10:31.706400 (+ 1808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:31.706405 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:31.706483 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:31.706586 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:31.708026 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:31.708776 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:31.710262 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:31.710660 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:31.710766 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:31.713967 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:31.713972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:31.713975 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:31.714008 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:31.716659 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:31.717307 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:31.717311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:31.717313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:31.717422 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:31.717816 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:31.717821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:31.717824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:31.718068 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:31.718220 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:31.718836 (+ 616us) server_negotiation.cc:299] Negotiation successful
0812 07:10:31.719043 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":227,"threads_started":1}
I20260812 07:10:31.720286 5755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:31.703626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:40205)
0812 07:10:31.704449 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:31.704496 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:31.706156 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:31.706830 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:31.706838 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:31.707338 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:31.707868 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:31.707885 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:31.708932 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:31.708936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:31.710127 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:31.710140 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:31.710869 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:31.710877 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:31.711109 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:31.711698 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:31.711722 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:31.713770 (+ 2048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:31.716781 (+ 3011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:31.716789 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:31.716803 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:31.717196 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:31.717524 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:31.717529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:31.717533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:31.717711 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:31.718192 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:31.718199 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:31.718493 (+ 294us) client_negotiation.cc:769] Sending connection context
0812 07:10:31.718717 (+ 224us) client_negotiation.cc:240] Negotiation successful
0812 07:10:31.719201 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":329,"threads_started":1}
I20260812 07:10:31.721753 5749 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37881
I20260812 07:10:31.722023 5749 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:31.722676 5749 heartbeater.cc:511] Master 127.21.181.126:37881 requested a full tablet report, sending...
I20260812 07:10:31.724670 5384 ts_manager.cc:194] Registered new tserver with Master: 266ccc3a364c4c98a30ded3e1a0fb047 (127.21.181.65:42451)
I20260812 07:10:31.726052 5384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:40205
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:10:31.756451 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37881
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:32.074568 5758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:32.074844 5758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:32.074944 5758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:32.082109 5758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:32.082217 5758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:32.082327 5758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:32.091645 5758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:32.092859 5758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:32.094094 5758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:32.106436 5769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:32.106669 5773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:32.107007 5767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:32.107748 5758 server_base.cc:1034] running on GCE node
I20260812 07:10:32.108359 5758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:32.109231 5758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:32.110428 5758 hybrid_clock.cc:648] HybridClock initialized: now 1786518632110376 us; error 45 us; skew 500 ppm
I20260812 07:10:32.110716 5758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:32 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:32.113965 5758 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:10:32.115518 5758 webserver.cc:460] Webserver started at http://127.21.181.66:45377/ using document root <none> and password file <none>
I20260812 07:10:32.116394 5758 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:32.116519 5758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:32.116863 5758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:32.120579 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "3db42280dc314f6e9c9a314e86bcca7c"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "06cf4b2453cad282fb6326779c50ab3d"
server_key_iv: "663575acfef09b04e0ac786953b79cf7"
server_key_version: "encryptionkey@0"
I20260812 07:10:32.121492 5758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "3db42280dc314f6e9c9a314e86bcca7c"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "06cf4b2453cad282fb6326779c50ab3d"
server_key_iv: "663575acfef09b04e0ac786953b79cf7"
server_key_version: "encryptionkey@0"
I20260812 07:10:32.126331 5758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:10:32.129617 5788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:32.130859 5758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:10:32.131016 5758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "3db42280dc314f6e9c9a314e86bcca7c"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "06cf4b2453cad282fb6326779c50ab3d"
server_key_iv: "663575acfef09b04e0ac786953b79cf7"
server_key_version: "encryptionkey@0"
I20260812 07:10:32.131196 5758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:32.151554 5758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:32.155531 5758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:32.155838 5758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:32.156852 5758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:32.158249 5758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:32.158331 5758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:32.158461 5758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:32.158519 5758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:32.190037 5758 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:40579
I20260812 07:10:32.190104 6003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:40579 every 8 connection(s)
I20260812 07:10:32.191303 5758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:32.197156 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 5758
I20260812 07:10:32.197350 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:32.197779 22229 external_mini_cluster.cc:1428] Setting key 2ce5610e79e0f8a8d1490c5db67a8117
Aug 12 07:10:32 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:32.210799 5754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:32.193770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:45177 (local address 127.21.181.126:37881)
0812 07:10:32.193942 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:32.193947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:32.195134 (+ 1187us) server_negotiation.cc:407] Connection header received
0812 07:10:32.196939 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:32.196944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:32.197036 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:32.197157 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:32.198782 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.199634 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.201185 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.201593 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.201704 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:32.205208 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:32.205214 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:32.205218 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:32.205259 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:32.208106 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.208771 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.208776 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.208779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.208934 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.209303 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.209307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.209311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.209620 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:32.209745 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:32.210386 (+ 641us) server_negotiation.cc:299] Negotiation successful
0812 07:10:32.210566 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:10:32.212015 6006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:32.194250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:45177)
0812 07:10:32.194963 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:32.195010 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:32.196664 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:32.197356 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:32.197363 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:32.197941 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:32.198609 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:32.198630 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.199791 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.199795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:32.201045 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:32.201057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.201820 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.201829 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:32.202086 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:32.202691 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:32.202726 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:32.204937 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:32.208228 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:32.208235 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:32.208247 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:32.208654 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:32.209026 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:32.209029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:32.209032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:32.209204 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:32.209721 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:32.209726 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:32.210028 (+ 302us) client_negotiation.cc:769] Sending connection context
0812 07:10:32.210239 (+ 211us) client_negotiation.cc:240] Negotiation successful
0812 07:10:32.210697 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":267,"threads_started":1}
I20260812 07:10:32.213368 6004 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37881
I20260812 07:10:32.213647 6004 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:32.214331 6004 heartbeater.cc:511] Master 127.21.181.126:37881 requested a full tablet report, sending...
I20260812 07:10:32.215747 5384 ts_manager.cc:194] Registered new tserver with Master: 3db42280dc314f6e9c9a314e86bcca7c (127.21.181.66:40579)
I20260812 07:10:32.216795 5384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:45177
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:10:32.252139 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37881
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:32.567761 6014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:32.568034 6014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:32.568195 6014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:32.575163 6014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:32.575274 6014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:32.575423 6014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:32.584688 6014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:32.585879 6014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:32.587126 6014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:32.599821 6023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:32.599828 6024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:32.600266 6026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:32.600422 6014 server_base.cc:1034] running on GCE node
I20260812 07:10:32.601022 6014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:32.601905 6014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:32.603104 6014 hybrid_clock.cc:648] HybridClock initialized: now 1786518632603052 us; error 44 us; skew 500 ppm
I20260812 07:10:32.603395 6014 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:32 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:32.606330 6014 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:10:32.607703 6014 webserver.cc:460] Webserver started at http://127.21.181.67:46263/ using document root <none> and password file <none>
I20260812 07:10:32.608389 6014 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:32.608507 6014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:32.608793 6014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:32.611582 6014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "532eb89bafcf4ba2897c86954a087792"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "2309aa792ea711092ccb21b71fcb0eed"
server_key_iv: "e971237dabf8ad132020d5351bc3b6e3"
server_key_version: "encryptionkey@0"
I20260812 07:10:32.612380 6014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "532eb89bafcf4ba2897c86954a087792"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "2309aa792ea711092ccb21b71fcb0eed"
server_key_iv: "e971237dabf8ad132020d5351bc3b6e3"
server_key_version: "encryptionkey@0"
I20260812 07:10:32.617338 6014 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:10:32.620602 6034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:32.621809 6014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:10:32.621950 6014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "532eb89bafcf4ba2897c86954a087792"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "2309aa792ea711092ccb21b71fcb0eed"
server_key_iv: "e971237dabf8ad132020d5351bc3b6e3"
server_key_version: "encryptionkey@0"
I20260812 07:10:32.622128 6014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:32.634163 6014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:32.638317 6014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:32.638573 6014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:32.639410 6014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:32.640664 6014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:32.640743 6014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:32.640856 6014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:32.640910 6014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:32.670631 6014 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:39417
I20260812 07:10:32.670674 6236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:39417 every 8 connection(s)
I20260812 07:10:32.671838 6014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:32.682168 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 6014
I20260812 07:10:32.682385 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:32.682909 22229 external_mini_cluster.cc:1428] Setting key 09238053048d3b2306e10b9d35e124c7
Aug 12 07:10:32 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518632, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:32.690478 5754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:32.674207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:41009 (local address 127.21.181.126:37881)
0812 07:10:32.674379 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:32.674383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:32.675559 (+ 1176us) server_negotiation.cc:407] Connection header received
0812 07:10:32.677348 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:32.677352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:32.677474 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:32.677568 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:32.679010 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.679755 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.681252 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.681659 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.681752 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:32.685308 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:32.685313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:32.685317 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:32.685347 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:32.687682 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.688386 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.688390 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.688393 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.688523 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.688911 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.688915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.688917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.689174 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:32.689303 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:32.689933 (+ 630us) server_negotiation.cc:299] Negotiation successful
0812 07:10:32.690145 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:10:32.691282 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:32.674695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:41009)
0812 07:10:32.675401 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:32.675436 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:32.677112 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:32.677790 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:32.677800 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:32.678318 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:32.678857 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:32.678873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.679860 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.679863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:32.681106 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:32.681123 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.681852 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.681860 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:32.682165 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:32.682825 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:32.682853 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:32.685119 (+ 2266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:32.687807 (+ 2688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:32.687816 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:32.687830 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:32.688271 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:32.688624 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:32.688628 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:32.688631 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:32.688799 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:32.689295 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:32.689302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:32.689577 (+ 275us) client_negotiation.cc:769] Sending connection context
0812 07:10:32.689802 (+ 225us) client_negotiation.cc:240] Negotiation successful
0812 07:10:32.690274 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":289,"threads_started":1}
I20260812 07:10:32.692432 6237 heartbeater.cc:347] Connected to a master server at 127.21.181.126:37881
I20260812 07:10:32.692684 6237 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:32.693351 6237 heartbeater.cc:511] Master 127.21.181.126:37881 requested a full tablet report, sending...
I20260812 07:10:32.694581 5384 ts_manager.cc:194] Registered new tserver with Master: 532eb89bafcf4ba2897c86954a087792 (127.21.181.67:39417)
I20260812 07:10:32.695319 5384 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:41009
I20260812 07:10:32.697901 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.65@KRBTEST.COM" created.
I20260812 07:10:32.728807 5749 heartbeater.cc:503] Master 127.21.181.126:37881 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab.
Entry for principal oryx/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab.
WARNING: no policy specified for oryx/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab.
Entry for principal oryx/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab.
WARNING: no policy specified for oryx/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab.
Entry for principal oryx/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab.
I20260812 07:10:32.833884 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 5504
Aug 12 07:10:32 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:10:32.849352 6260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:32.835058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58518 (local address 127.21.181.65:42451)
0812 07:10:32.835760 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:32.835767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:32.835795 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:10:32.835859 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:32.835865 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:32.836117 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:32.836261 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:32.837480 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.838459 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.839638 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.840226 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.840319 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:32.844234 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:32.844246 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:32.844260 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:32.844294 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:32.846999 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.847562 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.847568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.847571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.847689 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.847974 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.847979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.847981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.848359 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:32.848596 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:32.848866 (+ 270us) server_negotiation.cc:299] Negotiation successful
0812 07:10:32.849067 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":370,"threads_started":1}
I20260812 07:10:32.850598 5738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:32.930971 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 5504
I20260812 07:10:32.956570 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 5758
Aug 12 07:10:32 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:10:32.971212 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:32.957486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60894 (local address 127.21.181.66:40579)
0812 07:10:32.958004 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:32.958012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:32.958039 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:10:32.958102 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:32.958109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:32.958321 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:32.958450 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:32.959567 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.960606 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.961769 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:32.962293 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:32.962383 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:32.966228 (+ 3845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:32.966239 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:32.966253 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:32.966285 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:32.968988 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.969528 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.969534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.969536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.969652 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:32.969901 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:32.969903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:32.969905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:32.970232 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:32.970458 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:32.970658 (+ 200us) server_negotiation.cc:299] Negotiation successful
0812 07:10:32.970841 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":232,"threads_started":1}
I20260812 07:10:32.972409 5995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:33.051848 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 5758
I20260812 07:10:33.070602 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6014
Aug 12 07:10:33 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518630, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:10:33.086217 6267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:33.071551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51732 (local address 127.21.181.67:39417)
0812 07:10:33.072339 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:33.072346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:33.072376 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:10:33.072436 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:33.072443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:33.072653 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:33.072796 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:33.074156 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:33.075227 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:33.076527 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:33.077102 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:33.077227 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:33.081389 (+ 4162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:33.081401 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:33.081415 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:33.081449 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:33.083936 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:33.084548 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:33.084555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:33.084557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:33.084679 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:33.084969 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:33.084972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:33.084974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:33.085305 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:33.085545 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:33.085724 (+ 179us) server_negotiation.cc:299] Negotiation successful
0812 07:10:33.085920 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":333,"threads_started":1}
I20260812 07:10:33.087446 6231 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:33.169160 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 6014
I20260812 07:10:33.193866 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 5319
I20260812 07:10:33.195005 5444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:33.296959 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 5319
I20260812 07:10:33.325779 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37881
--webserver_interface=127.21.181.126
--webserver_port=39797
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:37881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:33.645623 6271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:33.645890 6271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:33.646026 6271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:33.653024 6271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:33.653119 6271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:33.653191 6271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:33.653265 6271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:33.653321 6271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:33.662642 6271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:37881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=39797
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:33.663841 6271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:33.665114 6271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:33.676035 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:33.675992 6280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:33.676069 6288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:33.676838 6271 server_base.cc:1034] running on GCE node
I20260812 07:10:33.677578 6271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:33.678881 6271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:33.680094 6271 hybrid_clock.cc:648] HybridClock initialized: now 1786518633680031 us; error 37 us; skew 500 ppm
I20260812 07:10:33.680431 6271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:33 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518633, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:33.683243 6271 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:10:33.684661 6271 webserver.cc:460] Webserver started at http://127.21.181.126:39797/ using document root <none> and password file <none>
I20260812 07:10:33.685294 6271 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:33.685424 6271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:33.690524 6271 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:10:33.693104 6297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:33.694319 6271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:33.694483 6271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "15f2b666c83a4762a3a2d39cd28afc58"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "9d206c0edfa78c74bbe7c0a0b3017110"
server_key_iv: "8df44d0dda764ffa07fa37e449b1d903"
server_key_version: "encryptionkey@0"
I20260812 07:10:33.695427 6271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:33.707494 6271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:33.711042 6271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:33.711267 6271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:33.728163 6271 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:37881
I20260812 07:10:33.728184 6394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:37881 every 8 connection(s)
I20260812 07:10:33.729625 6271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:33.736245 6395 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:10:33.738942 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 6271
I20260812 07:10:33.739351 6395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Bootstrap starting.
I20260812 07:10:33.739719 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:42451
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:37881
--webserver_port=43435
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:10:33.757774 6395 log.cc:824] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:33.766366 6395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:10:33.766844 6395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Bootstrap complete.
I20260812 07:10:33.775746 6395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:33.776610 6395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15f2b666c83a4762a3a2d39cd28afc58, State: Initialized, Role: FOLLOWER
I20260812 07:10:33.777189 6395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:33.777432 6395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:33.777539 6395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:33.777695 6395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:10:33.781373 6395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:33.781850 6395 leader_election.cc:302] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15f2b666c83a4762a3a2d39cd28afc58; no voters:
I20260812 07:10:33.782397 6395 leader_election.cc:288] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:10:33.782575 6400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:10:33.783514 6400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [term 2 LEADER]: Becoming Leader. State: Replica: 15f2b666c83a4762a3a2d39cd28afc58, State: Running, Role: LEADER
I20260812 07:10:33.784307 6395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:33.784211 6400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } }
I20260812 07:10:33.786780 6404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15f2b666c83a4762a3a2d39cd28afc58. Latest consensus state: current_term: 2 leader_uuid: "15f2b666c83a4762a3a2d39cd28afc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } } }
I20260812 07:10:33.787202 6404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:33.787665 6402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "15f2b666c83a4762a3a2d39cd28afc58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15f2b666c83a4762a3a2d39cd28afc58" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37881 } } }
I20260812 07:10:33.787961 6402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:33.794981 6412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:33.801450 6412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:33.803001 6412 catalog_manager.cc:1259] Loaded cluster ID: c0b56e25862a4d8099356d5d397724c3
I20260812 07:10:33.803185 6412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:33.807350 6412 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:33.808929 6412 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 15f2b666c83a4762a3a2d39cd28afc58: Loaded TSK: 0
I20260812 07:10:33.810307 6425 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:10:33.810631 6412 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:10:34.108188 6397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:34.108472 6397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:34.108611 6397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:34.115660 6397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:34.115756 6397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:34.115893 6397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:34.125401 6397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.65.keytab
--principal=oryx/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:42451
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=43435
--tserver_master_addrs=127.21.181.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:34.126650 6397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:34.127925 6397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:34.141729 6433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:34.142025 6440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:34.141763 6437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:34.142915 6397 server_base.cc:1034] running on GCE node
I20260812 07:10:34.143492 6397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:34.144441 6397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:34.145644 6397 hybrid_clock.cc:648] HybridClock initialized: now 1786518634145586 us; error 49 us; skew 500 ppm
I20260812 07:10:34.145953 6397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518634, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:34.148932 6397 init.cc:377] Logged in from keytab as oryx/127.21.181.65@KRBTEST.COM (short username oryx)
I20260812 07:10:34.150341 6397 webserver.cc:460] Webserver started at http://127.21.181.65:43435/ using document root <none> and password file <none>
I20260812 07:10:34.150985 6397 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:34.151103 6397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:34.156558 6397 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:10:34.159235 6450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:34.160670 6397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:10:34.160840 6397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "266ccc3a364c4c98a30ded3e1a0fb047"
format_stamp: "Formatted at 2026-08-12 07:10:31 on dist-test-slave-lgr5"
server_key: "e17c8cec2e4f0cee5d9f590afda4135b"
server_key_iv: "66cb926bba828c5e1a4b47b57857d3ac"
server_key_version: "encryptionkey@0"
I20260812 07:10:34.161787 6397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:34.237808 6397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:34.242027 6397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:34.242288 6397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:34.243251 6397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:34.244598 6397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:34.244696 6397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:34.244796 6397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:34.244874 6397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:34.274909 6397 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:42451
I20260812 07:10:34.274963 6680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:42451 every 8 connection(s)
I20260812 07:10:34.276327 6397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:34.286868 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 6397
I20260812 07:10:34.287515 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:40579
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:37881
--webserver_port=45377
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:34.295890 6687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:34.279237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:59039)
0812 07:10:34.279992 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:34.280038 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:34.281696 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:34.282873 (+ 1177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:34.282880 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:34.283389 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:34.283908 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.283925 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.285508 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.285514 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.287039 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.287062 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.288221 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.288257 (+ 36us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.288580 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.289812 (+ 1232us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:34.289840 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:34.294358 (+ 4518us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":308,"threads_started":1}
W20260812 07:10:34.296254 6686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:34.279080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:59039 (local address 127.21.181.126:37881)
0812 07:10:34.279751 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:34.279766 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:34.280177 (+ 411us) server_negotiation.cc:407] Connection header received
0812 07:10:34.282171 (+ 1994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:34.282183 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:34.282485 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:34.282759 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:34.284131 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.285368 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.287176 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.287922 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.288116 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.294951 (+ 6835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:59039: BlockingRecv error: recv got EOF from 127.21.181.65:59039 (error 108)
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":221,"threads_started":1}
W20260812 07:10:34.296836 6683 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:37881 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.181.126:37881: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:34.307016 6687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:34.297469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:47337)
0812 07:10:34.297706 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:34.297725 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:34.297824 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:34.298180 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:34.298184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:34.298538 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:34.298706 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.298717 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.300062 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.300068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.301085 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.301097 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.302556 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.302563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.302763 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.304132 (+ 1369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:34.304151 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:34.306779 (+ 2628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113}
W20260812 07:10:34.307853 6686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:34.297698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:47337 (local address 127.21.181.126:37881)
0812 07:10:34.297868 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:34.297874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:34.297892 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:10:34.297944 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:34.297950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:34.298020 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:34.298120 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:34.298906 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.299842 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.301554 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.302192 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.302328 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.307629 (+ 5301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:47337: BlockingRecv error: recv got EOF from 127.21.181.65:47337 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:34.318027 6690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:34.307645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:51329)
0812 07:10:34.308215 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:34.308232 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:34.308349 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:34.308704 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:34.308709 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:34.309215 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:34.309527 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.309539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.310847 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.310852 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.312414 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.312431 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.313351 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.313357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.313539 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.314198 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:34.314220 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:34.317707 (+ 3487us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":600,"threads_started":1}
W20260812 07:10:34.318225 6686 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:34.308006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:51329 (local address 127.21.181.126:37881)
0812 07:10:34.308194 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:34.308200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:34.308441 (+ 241us) server_negotiation.cc:407] Connection header received
0812 07:10:34.308493 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:34.308498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:34.308563 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:34.308646 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:34.309686 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.310714 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.312605 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.313184 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.313314 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.317935 (+ 4621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:51329: BlockingRecv error: recv got EOF from 127.21.181.65:51329 (error 108)
Metrics: {"server-negotiator.queue_time_us":42,"threads_started":1}
W20260812 07:10:34.318506 6683 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:10:34.618242 6689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:34.618551 6689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:34.618695 6689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:34.625756 6689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:34.625877 6689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:34.626032 6689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:34.635453 6689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.66.keytab
--principal=oryx/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:40579
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=45377
--tserver_master_addrs=127.21.181.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:34.636714 6689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:34.637926 6689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:34.650002 6703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:34.649937 6698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:34.650009 6697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:34.650838 6689 server_base.cc:1034] running on GCE node
I20260812 07:10:34.651394 6689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:34.652246 6689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:34.653439 6689 hybrid_clock.cc:648] HybridClock initialized: now 1786518634653392 us; error 41 us; skew 500 ppm
I20260812 07:10:34.653720 6689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518634, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:34.656559 6689 init.cc:377] Logged in from keytab as oryx/127.21.181.66@KRBTEST.COM (short username oryx)
I20260812 07:10:34.657800 6689 webserver.cc:460] Webserver started at http://127.21.181.66:45377/ using document root <none> and password file <none>
I20260812 07:10:34.658419 6689 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:34.658577 6689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:34.663486 6689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:10:34.665992 6715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:34.667184 6689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:10:34.667339 6689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "3db42280dc314f6e9c9a314e86bcca7c"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "06cf4b2453cad282fb6326779c50ab3d"
server_key_iv: "663575acfef09b04e0ac786953b79cf7"
server_key_version: "encryptionkey@0"
I20260812 07:10:34.668381 6689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:34.683434 6689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:34.687049 6689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:34.687259 6689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:34.688141 6689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:34.689391 6689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:34.689474 6689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:34.689553 6689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:34.689612 6689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:34.719538 6689 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:40579
I20260812 07:10:34.719583 6902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:40579 every 8 connection(s)
I20260812 07:10:34.720875 6689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:34.729923 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 6689
I20260812 07:10:34.730572 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:39417
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:37881
--webserver_port=46263
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:10:34.737056 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:34.723181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:49905 (local address 127.21.181.126:37881)
0812 07:10:34.723384 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:34.723390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:34.724491 (+ 1101us) server_negotiation.cc:407] Connection header received
0812 07:10:34.726187 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:34.726191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:34.726263 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:34.726354 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:34.727758 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.728568 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.730392 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.731101 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.731222 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.736789 (+ 5567us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:49905: BlockingRecv error: recv got EOF from 127.21.181.66:49905 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:10:34.737632 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:34.723632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:49905)
0812 07:10:34.724341 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:34.724379 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:34.725949 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:34.726533 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:34.726540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:34.727038 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:34.727593 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.727612 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.728723 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.728727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.730206 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.730222 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.731332 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.731339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.731606 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.732357 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:34.732384 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:34.736528 (+ 4144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":272,"threads_started":1}
W20260812 07:10:34.738289 6904 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:37881 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.181.126:37881: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:34.745625 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:34.738817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:38113)
0812 07:10:34.738969 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:34.738986 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:34.739071 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:34.739376 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:34.739381 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:34.739687 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:34.739810 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.739818 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.740839 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.740845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.741687 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.741698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.742307 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.742313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.742472 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.742945 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:34.742962 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:34.745446 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 07:10:34.745882 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:34.738995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:38113 (local address 127.21.181.126:37881)
0812 07:10:34.739154 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:34.739159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:34.739174 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:10:34.739214 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:34.739217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:34.739266 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:34.739348 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:34.739950 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.740698 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.741814 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.742183 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.742284 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.745688 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:38113: BlockingRecv error: recv got EOF from 127.21.181.66:38113 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:34 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:34.754046 6906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:34.746178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:33657)
0812 07:10:34.746467 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:34.746499 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:34.746675 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:34.747037 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:34.747041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:34.747356 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:34.747537 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.747548 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.748469 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.748482 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.749767 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:34.749782 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.750388 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.750393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:34.750580 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.751094 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:34.751114 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:34.753765 (+ 2651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":399,"threads_started":1}
W20260812 07:10:34.754164 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:34.746249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:33657 (local address 127.21.181.126:37881)
0812 07:10:34.746420 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:34.746426 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:34.746570 (+ 144us) server_negotiation.cc:407] Connection header received
0812 07:10:34.746858 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:34.746862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:34.746914 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:34.746985 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:34.747643 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.748335 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.749878 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:34.750258 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:34.750340 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:34.753959 (+ 3619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:33657: BlockingRecv error: recv got EOF from 127.21.181.66:33657 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 07:10:34.754303 6904 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:10:35.052107 6907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:35.052366 6907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:35.052467 6907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:35.059337 6907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:35.059428 6907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:35.059541 6907 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:35.068754 6907 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/krb5kdc/oryx_127.21.181.67.keytab
--principal=oryx/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:39417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=46263
--tserver_master_addrs=127.21.181.126:37881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:35.069938 6907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:35.071148 6907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:35.083235 6926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:35.083277 6925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:35.083192 6929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:35.084327 6907 server_base.cc:1034] running on GCE node
I20260812 07:10:35.084923 6907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:35.085836 6907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:35.087047 6907 hybrid_clock.cc:648] HybridClock initialized: now 1786518635087000 us; error 36 us; skew 500 ppm
I20260812 07:10:35.087329 6907 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518635, etypes {rep=17 tkt=17 ses=17}, oryx/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:35.090240 6907 init.cc:377] Logged in from keytab as oryx/127.21.181.67@KRBTEST.COM (short username oryx)
I20260812 07:10:35.091524 6907 webserver.cc:460] Webserver started at http://127.21.181.67:46263/ using document root <none> and password file <none>
I20260812 07:10:35.092185 6907 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:35.092319 6907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:35.097374 6907 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 07:10:35.099920 6944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:35.101150 6907 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:10:35.101298 6907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "532eb89bafcf4ba2897c86954a087792"
format_stamp: "Formatted at 2026-08-12 07:10:32 on dist-test-slave-lgr5"
server_key: "2309aa792ea711092ccb21b71fcb0eed"
server_key_iv: "e971237dabf8ad132020d5351bc3b6e3"
server_key_version: "encryptionkey@0"
I20260812 07:10:35.102206 6907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:35.126473 6907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:35.130002 6907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:35.130239 6907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:35.131170 6907 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:35.132426 6907 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:35.132539 6907 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:35.132640 6907 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:35.132711 6907 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:35.162379 6907 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:39417
I20260812 07:10:35.162441 7139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:39417 every 8 connection(s)
I20260812 07:10:35.163810 6907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:35.167953 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 6907
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
W20260812 07:10:35.179617 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:35.166269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:56743 (local address 127.21.181.126:37881)
0812 07:10:35.166469 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:35.166476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:35.167613 (+ 1137us) server_negotiation.cc:407] Connection header received
0812 07:10:35.169920 (+ 2307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:35.169925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:35.170006 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:35.170077 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:35.171596 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.172448 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.173979 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.174456 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.174562 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.179420 (+ 4858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:56743: BlockingRecv error: recv got EOF from 127.21.181.67:56743 (error 108)
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":61}
I20260812 07:10:35.180630 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:35.166762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:56743)
0812 07:10:35.167434 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:35.167480 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:35.169684 (+ 2204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:35.170342 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:35.170350 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:35.170839 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:35.171429 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.171446 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.172592 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.172596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.173843 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.173856 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.174667 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.174675 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.174873 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.175469 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:35.175499 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:35.179282 (+ 3783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":249,"threads_started":1}
W20260812 07:10:35.181218 7141 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:37881 (0 consecutive failures): Not authorized: Failed to ping master at 127.21.181.126:37881: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
I20260812 07:10:35.184365 7147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:35.169033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.21.181.126:37881)
0812 07:10:35.169670 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:35.169676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:35.169752 (+ 76us) server_negotiation.cc:407] Connection header received
0812 07:10:35.169971 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:35.169976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:35.170052 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:35.170162 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:35.171781 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.172614 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.173936 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.174382 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.174492 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.177033 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:35.177040 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:35.177054 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:35.177092 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:35.180586 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:35.181544 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:35.181550 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:35.181557 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:35.181716 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:35.182415 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:35.182421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:35.182424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:35.183015 (+ 591us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:35.183224 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:35.183682 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 07:10:35.183977 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":534,"thread_start_us":496,"threads_started":1}
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:35.189502 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:35.181722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:42835)
0812 07:10:35.181877 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:35.181894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:35.181983 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:35.182256 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:35.182260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:35.182607 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:35.182783 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.182794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.184112 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.184117 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.185299 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.185313 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.186095 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.186101 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.186267 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.186750 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:35.186766 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:35.189318 (+ 2552us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:10:35.189679 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:35.181833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:42835 (local address 127.21.181.126:37881)
0812 07:10:35.181993 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:35.181998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:35.182014 (+ 16us) server_negotiation.cc:407] Connection header received
0812 07:10:35.182069 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:35.182073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:35.182135 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:35.182223 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:35.182915 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.183930 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.185439 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.185946 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.186043 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.189489 (+ 3446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:42835: BlockingRecv error: recv got EOF from 127.21.181.67:42835 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:35.196257 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:35.190038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:45529)
0812 07:10:35.190176 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:35.190190 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:35.190259 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:35.190527 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:35.190530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:35.190767 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:35.190897 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.190904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.191722 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.191726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.192621 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.192632 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.193208 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.193211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.193319 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.193721 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:35.193736 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:35.196007 (+ 2271us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:10:35.196393 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:35.190124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:45529 (local address 127.21.181.126:37881)
0812 07:10:35.190329 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:35.190333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:35.190347 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:10:35.190382 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:35.190385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:35.190431 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:35.190508 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:35.190994 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.191624 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.192720 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.193111 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.193195 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.196220 (+ 3025us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:45529: BlockingRecv error: recv got EOF from 127.21.181.67:45529 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260812 07:10:35.196525 7141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:35.327263 7148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:35.319050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:47523)
0812 07:10:35.319528 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:35.319543 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:35.319682 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:35.319984 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:35.319989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:35.320474 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:35.320663 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.320673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.321670 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.321674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.322624 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.322637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.323360 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.323366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.323639 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.324209 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:35.324228 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:35.326951 (+ 2723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
W20260812 07:10:35.327447 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:35.319130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:47523 (local address 127.21.181.126:37881)
0812 07:10:35.319325 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:35.319331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:35.319628 (+ 297us) server_negotiation.cc:407] Connection header received
0812 07:10:35.319796 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:35.319801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:35.319860 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:35.319938 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:35.320809 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.321541 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.322757 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.323189 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.323310 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.327193 (+ 3883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:47523: BlockingRecv error: recv got EOF from 127.21.181.65:47523 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:35 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:35.762632 7150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:35.754835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:39653)
0812 07:10:35.755307 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:35.755323 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:35.755447 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:35.755762 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:35.755767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:35.756266 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:35.756461 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.756472 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.757451 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.757455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.758380 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:35.758392 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.759091 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.759097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:35.759241 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.759754 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:35.759770 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:35.762315 (+ 2545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":243,"threads_started":1}
W20260812 07:10:35.762741 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:35.754912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:39653 (local address 127.21.181.126:37881)
0812 07:10:35.755142 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:35.755149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:35.755398 (+ 249us) server_negotiation.cc:407] Connection header received
0812 07:10:35.755557 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:35.755562 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:35.755624 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:35.755702 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:35.756617 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.757320 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.758513 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:35.758912 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:35.759029 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:35.762518 (+ 3489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:39653: BlockingRecv error: recv got EOF from 127.21.181.66:39653 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 12 07:10:36 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:36 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:36.205219 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:36.197068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:33691)
0812 07:10:36.197567 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:36.197583 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:36.197694 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:36.197944 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:36.197947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:36.198451 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:36.198698 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:36.198709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.199969 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.199974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:36.200962 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:36.200977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.201641 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.201646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:36.201768 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:36.202288 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:36.202305 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:36.205000 (+ 2695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":253,"threads_started":1}
W20260812 07:10:36.205341 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:36.197118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:33691 (local address 127.21.181.126:37881)
0812 07:10:36.197274 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:36.197279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:36.197644 (+ 365us) server_negotiation.cc:407] Connection header received
0812 07:10:36.197780 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:36.197784 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:36.197841 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:36.197917 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:36.198841 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.199813 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.201091 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.201478 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.201579 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:36.205163 (+ 3584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:33691: BlockingRecv error: recv got EOF from 127.21.181.67:33691 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 07:10:36 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:36 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:36.336040 7153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:36.328266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:34925)
0812 07:10:36.328741 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:36.328757 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:36.328884 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:36.329214 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:36.329219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:36.329643 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:36.329803 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:36.329812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.330788 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.330792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:36.331673 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:36.331685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.332435 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.332441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:36.332601 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:36.333183 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:36.333201 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:36.335774 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":250,"threads_started":1}
W20260812 07:10:36.336196 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:36.328305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:34925 (local address 127.21.181.126:37881)
0812 07:10:36.328485 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:36.328492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:36.328834 (+ 342us) server_negotiation.cc:407] Connection header received
0812 07:10:36.329013 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:36.329019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:36.329085 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:36.329168 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:36.329938 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.330654 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.331803 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.332281 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.332385 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:36.335968 (+ 3583us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:34925: BlockingRecv error: recv got EOF from 127.21.181.65:34925 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 07:10:36 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:36 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:36.774250 7154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:36.763677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:42569)
0812 07:10:36.764328 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:36.764344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:36.764454 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:36.764750 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:36.764753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:36.765170 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:36.765375 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:36.765387 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.766347 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.766352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:36.767248 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:36.767261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.767861 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.767865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:36.767970 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:36.771275 (+ 3305us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:36.771293 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:36.774016 (+ 2723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":396,"threads_started":1}
W20260812 07:10:36.774402 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:36.763731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:42569 (local address 127.21.181.126:37881)
0812 07:10:36.763933 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:36.763940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:36.764413 (+ 473us) server_negotiation.cc:407] Connection header received
0812 07:10:36.764553 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:36.764556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:36.764616 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:36.764699 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:36.765502 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.766210 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.767355 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:36.767752 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:36.767836 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:36.774174 (+ 6338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:42569: BlockingRecv error: recv got EOF from 127.21.181.66:42569 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 07:10:37 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:37 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:37.213874 7155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:37.206160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:52505)
0812 07:10:37.206674 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:37.206692 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:37.206804 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:37.207065 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:37.207069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:37.207485 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:37.207675 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:37.207688 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.208635 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.208638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:37.209557 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:37.209576 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.210375 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.210382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:37.210557 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:37.211136 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:37.211154 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:37.213644 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":260,"threads_started":1}
W20260812 07:10:37.214006 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:37.206276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:52505 (local address 127.21.181.126:37881)
0812 07:10:37.206430 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:37.206435 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:37.206762 (+ 327us) server_negotiation.cc:407] Connection header received
0812 07:10:37.206896 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:37.206899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:37.206959 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:37.207034 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:37.207794 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.208533 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.209694 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.210232 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.210356 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:37.213823 (+ 3467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:52505: BlockingRecv error: recv got EOF from 127.21.181.67:52505 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 12 07:10:37 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:37 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:37.344938 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:37.337041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:60807)
0812 07:10:37.337540 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:37.337556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:37.337686 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:37.337999 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:37.338005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:37.338357 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:37.338550 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:37.338560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.339575 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.339580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:37.340687 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:37.340700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.341370 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.341376 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:37.341546 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:37.342049 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:37.342066 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:37.344591 (+ 2525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":262,"threads_started":1}
W20260812 07:10:37.345050 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:37.337049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:60807 (local address 127.21.181.126:37881)
0812 07:10:37.337232 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:37.337238 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:37.337638 (+ 400us) server_negotiation.cc:407] Connection header received
0812 07:10:37.337796 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:37.337801 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:37.337869 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:37.337948 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:37.338694 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.339435 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.340816 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.341225 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.341332 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:37.344811 (+ 3479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:60807: BlockingRecv error: recv got EOF from 127.21.181.65:60807 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 07:10:37 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:37 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:37.782606 7158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:37.775069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:54171)
0812 07:10:37.775589 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:37.775605 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:37.775715 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:37.775970 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:37.775973 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:37.776313 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:37.776505 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:37.776514 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.777436 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.777439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:37.778327 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:37.778338 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.778916 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.778920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:37.779034 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:37.779557 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:37.779577 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:37.782342 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":270,"threads_started":1}
W20260812 07:10:37.782671 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:37.775121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:54171 (local address 127.21.181.126:37881)
0812 07:10:37.775269 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:37.775273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:37.775665 (+ 392us) server_negotiation.cc:407] Connection header received
0812 07:10:37.775812 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:37.775815 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:37.775873 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:37.775942 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:37.776618 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.777339 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.778423 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:37.778820 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:37.778901 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:37.782508 (+ 3607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:54171: BlockingRecv error: recv got EOF from 127.21.181.66:54171 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 07:10:38 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:38 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:38.222698 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:38.214706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:41469)
0812 07:10:38.215226 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:38.215242 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:38.215372 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:38.215711 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:38.215716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:38.216116 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:38.216289 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:38.216298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.217308 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.217312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:38.218335 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:38.218353 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.219240 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.219245 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:38.219375 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:38.219910 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:38.219928 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:38.222448 (+ 2520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":280,"threads_started":1}
W20260812 07:10:38.222858 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:38.214791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:41469 (local address 127.21.181.126:37881)
0812 07:10:38.214981 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:38.214988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:38.215327 (+ 339us) server_negotiation.cc:407] Connection header received
0812 07:10:38.215490 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:38.215502 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:38.215577 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:38.215656 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:38.216436 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.217167 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.218489 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.219103 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.219197 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:38.222640 (+ 3443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:41469: BlockingRecv error: recv got EOF from 127.21.181.67:41469 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 07:10:38 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:38 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:38.353082 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:38.345841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:43195)
0812 07:10:38.346277 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:38.346292 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:38.346406 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:38.346683 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:38.346686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:38.347050 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:38.347199 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:38.347208 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.348104 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.348107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:38.349082 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:38.349094 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.349655 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.349659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:38.349775 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:38.350288 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:38.350305 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:38.352862 (+ 2557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":228,"threads_started":1}
W20260812 07:10:38.353222 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:38.345879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:43195 (local address 127.21.181.126:37881)
0812 07:10:38.346081 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:38.346088 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:38.346363 (+ 275us) server_negotiation.cc:407] Connection header received
0812 07:10:38.346516 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:38.346520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:38.346582 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:38.346651 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:38.347302 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.347977 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.349191 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.349554 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.349640 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:38.353029 (+ 3389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:43195: BlockingRecv error: recv got EOF from 127.21.181.65:43195 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 07:10:38 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:38 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:38.791121 7168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:38.783444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:46513)
0812 07:10:38.783937 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:38.783953 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:38.784106 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:38.784396 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:38.784399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:38.784831 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:38.785004 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:38.785013 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.786036 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.786040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:38.786997 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:38.787009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.787649 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.787653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:38.787767 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:38.788300 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:38.788317 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:38.790895 (+ 2578us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":263,"threads_started":1}
W20260812 07:10:38.791280 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:38.783511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:46513 (local address 127.21.181.126:37881)
0812 07:10:38.783677 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:38.783681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:38.784027 (+ 346us) server_negotiation.cc:407] Connection header received
0812 07:10:38.784210 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:38.784215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:38.784282 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:38.784360 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:38.785108 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.785925 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.787101 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:38.787532 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:38.787637 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:38.791078 (+ 3441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:46513: BlockingRecv error: recv got EOF from 127.21.181.66:46513 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 07:10:39 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:39 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:39.231278 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:39.223511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:42939)
0812 07:10:39.223941 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:39.223957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:39.224099 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:39.224364 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:39.224367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:39.224781 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:39.224935 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:39.224944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.225880 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.225883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:39.226851 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:39.226863 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.227448 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.227452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:39.227578 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:39.228146 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:39.228164 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:39.230968 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":225,"threads_started":1}
W20260812 07:10:39.231403 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:39.223574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:42939 (local address 127.21.181.126:37881)
0812 07:10:39.223769 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:39.223775 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:39.224038 (+ 263us) server_negotiation.cc:407] Connection header received
0812 07:10:39.224194 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:39.224197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:39.224257 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:39.224333 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:39.225050 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.225785 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.226958 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.227346 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.227432 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:39.231179 (+ 3747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:42939: BlockingRecv error: recv got EOF from 127.21.181.67:42939 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 07:10:39 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:39 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:39.361986 7172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:39.353937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:32775)
0812 07:10:39.354378 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:39.354394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:39.354506 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:39.354758 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:39.354761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:39.355176 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:39.355350 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:39.355359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.356339 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.356344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:39.357525 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:39.357539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.358227 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.358232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:39.358356 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:39.358851 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:39.358866 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:39.361668 (+ 2802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":216,"threads_started":1}
W20260812 07:10:39.362088 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:39.353999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:32775 (local address 127.21.181.126:37881)
0812 07:10:39.354208 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:39.354215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:39.354465 (+ 250us) server_negotiation.cc:407] Connection header received
0812 07:10:39.354593 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:39.354597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:39.354655 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:39.354729 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:39.355455 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.356209 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.357635 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.358093 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.358181 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:39.361878 (+ 3697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:32775: BlockingRecv error: recv got EOF from 127.21.181.65:32775 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:10:39 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:39 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:39.799738 7173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:39.791928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:44347)
0812 07:10:39.792420 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:39.792436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:39.792547 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:39.792892 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:39.792897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:39.793306 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:39.793474 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:39.793483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.794483 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.794487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:39.795465 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:39.795479 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.796149 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.796155 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:39.796307 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:39.796802 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:39.796818 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:39.799411 (+ 2593us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":248,"threads_started":1}
W20260812 07:10:39.799815 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:39.792093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:44347 (local address 127.21.181.126:37881)
0812 07:10:39.792282 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:39.792288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:39.792501 (+ 213us) server_negotiation.cc:407] Connection header received
0812 07:10:39.792665 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:39.792670 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:39.792750 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:39.792838 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:39.793617 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.794351 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.795599 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:39.795988 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:39.796120 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:39.799612 (+ 3492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:44347: BlockingRecv error: recv got EOF from 127.21.181.66:44347 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:10:40 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:40 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:40.239789 7174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:40.232263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:34299)
0812 07:10:40.232761 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:40.232777 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:40.232891 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:40.233152 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:40.233155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:40.233536 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:40.233703 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:40.233713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.234654 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.234658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:40.235586 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:40.235597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.236206 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.236210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:40.236321 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:40.236871 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:40.236888 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:40.239548 (+ 2660us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":238,"threads_started":1}
W20260812 07:10:40.239923 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:40.232353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:34299 (local address 127.21.181.126:37881)
0812 07:10:40.232515 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:40.232520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:40.232841 (+ 321us) server_negotiation.cc:407] Connection header received
0812 07:10:40.232980 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:40.232984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:40.233046 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:40.233122 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:40.233812 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.234552 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.235693 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.236108 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.236198 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:40.239716 (+ 3518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:34299: BlockingRecv error: recv got EOF from 127.21.181.67:34299 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 07:10:40 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:40 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:40.370898 7175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:40.362821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:45455)
0812 07:10:40.363281 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:40.363297 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:40.363412 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:40.363681 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:40.363684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:40.364070 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:40.364257 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:40.364267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.365227 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.365231 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:40.366208 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:40.366222 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.367034 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.367041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:40.367213 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:40.367830 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:40.367853 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:40.370685 (+ 2832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":258,"threads_started":1}
W20260812 07:10:40.371043 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:40.362870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:45455 (local address 127.21.181.126:37881)
0812 07:10:40.363074 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:40.363081 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:40.363368 (+ 287us) server_negotiation.cc:407] Connection header received
0812 07:10:40.363513 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:40.363516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:40.363579 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:40.363647 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:40.364366 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.365115 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.366326 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.366881 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.366990 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:40.370843 (+ 3853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:45455: BlockingRecv error: recv got EOF from 127.21.181.65:45455 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 12 07:10:40 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:40 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:40.808495 7177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:40.800698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:37597)
0812 07:10:40.801159 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:40.801176 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:40.801285 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:40.801544 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:40.801547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:40.801927 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:40.802080 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:40.802089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.803066 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.803069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:40.803964 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:40.803976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.804643 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.804647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:40.804764 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:40.805283 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:40.805300 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:40.808189 (+ 2889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":244,"threads_started":1}
W20260812 07:10:40.808636 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:40.800761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:37597 (local address 127.21.181.126:37881)
0812 07:10:40.800968 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:40.800974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:40.801250 (+ 276us) server_negotiation.cc:407] Connection header received
0812 07:10:40.801372 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:40.801375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:40.801437 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:40.801513 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:40.802202 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.802959 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.804113 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:40.804529 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:40.804618 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:40.808400 (+ 3782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:37597: BlockingRecv error: recv got EOF from 127.21.181.66:37597 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 12 07:10:41 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:41 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:41.248716 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:41.240682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:48485)
0812 07:10:41.241186 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:41.241202 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:41.241314 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:41.241608 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:41.241611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:41.241996 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:41.242164 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:41.242173 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.243082 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.243085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:41.243958 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:41.243969 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.244837 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.244844 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:41.245022 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:41.245631 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:41.245656 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:41.248424 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":292,"threads_started":1}
W20260812 07:10:41.248852 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:41.240734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:48485 (local address 127.21.181.126:37881)
0812 07:10:41.240938 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:41.240945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:41.241276 (+ 331us) server_negotiation.cc:407] Connection header received
0812 07:10:41.241408 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:41.241412 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:41.241502 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:41.241581 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:41.242278 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.242979 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.244097 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.244675 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.244771 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:41.248632 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:48485: BlockingRecv error: recv got EOF from 127.21.181.67:48485 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 07:10:41 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:41 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:41.379474 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:41.371772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:48625)
0812 07:10:41.372383 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:41.372398 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:41.372513 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:41.372816 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:41.372821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:41.373206 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:41.373373 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:41.373383 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.374344 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.374347 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:41.375274 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:41.375287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.375870 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.375874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:41.375980 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:41.376584 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:41.376602 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:41.379240 (+ 2638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":265,"threads_started":1}
W20260812 07:10:41.379621 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:41.371833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:48625 (local address 127.21.181.126:37881)
0812 07:10:41.372015 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:41.372019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:41.372462 (+ 443us) server_negotiation.cc:407] Connection header received
0812 07:10:41.372611 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:41.372614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:41.372673 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:41.372761 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:41.373496 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.374239 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.375378 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.375763 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.375847 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:41.379415 (+ 3568us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:48625: BlockingRecv error: recv got EOF from 127.21.181.65:48625 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:10:41 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:41 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:41.821705 7181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:41.809354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:58907)
0812 07:10:41.809865 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:41.809880 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:41.810027 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:41.810372 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:41.810377 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:41.810798 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:41.810970 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:41.810990 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.811998 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.812002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:41.812964 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:41.812985 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.813665 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.813671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:41.813826 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:41.818769 (+ 4943us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:41.818788 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:41.821362 (+ 2574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":227,"threads_started":1}
W20260812 07:10:41.821873 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:41.809408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:58907 (local address 127.21.181.126:37881)
0812 07:10:41.809629 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:41.809635 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:41.809968 (+ 333us) server_negotiation.cc:407] Connection header received
0812 07:10:41.810141 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:41.810147 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:41.810236 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:41.810317 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:41.811125 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.811847 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.813114 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:41.813522 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:41.813629 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:41.821644 (+ 8015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:58907: BlockingRecv error: recv got EOF from 127.21.181.66:58907 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 12 07:10:42 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:42 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:42.257526 7182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:42.249542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:51095)
0812 07:10:42.250058 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:42.250074 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:42.250206 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:42.250554 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:42.250559 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:42.250984 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:42.251153 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:42.251163 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.252189 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.252195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:42.253175 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:42.253189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.253910 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.253915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:42.254076 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:42.254622 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:42.254641 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:42.257173 (+ 2532us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":227,"threads_started":1}
W20260812 07:10:42.257644 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:42.249616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:51095 (local address 127.21.181.126:37881)
0812 07:10:42.249779 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:42.249786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:42.250157 (+ 371us) server_negotiation.cc:407] Connection header received
0812 07:10:42.250322 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:42.250328 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:42.250404 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:42.250485 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:42.251291 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.252021 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.253321 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.253760 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.253874 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:42.257393 (+ 3519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:51095: BlockingRecv error: recv got EOF from 127.21.181.67:51095 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 12 07:10:42 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:42 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:42.388495 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:42.380448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:39571)
0812 07:10:42.381165 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:42.381180 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:42.381310 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:42.381628 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:42.381633 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:42.382040 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:42.382209 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:42.382219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.383214 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.383220 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:42.384196 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:42.384209 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.384950 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.384956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:42.385110 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:42.385660 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:42.385678 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:42.388177 (+ 2499us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":449,"threads_started":1}
W20260812 07:10:42.388569 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:42.380628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:39571 (local address 127.21.181.126:37881)
0812 07:10:42.380817 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:42.380824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:42.381265 (+ 441us) server_negotiation.cc:407] Connection header received
0812 07:10:42.381423 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:42.381429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:42.381495 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:42.381573 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:42.382354 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.383076 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.384350 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.384773 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.384883 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:42.388359 (+ 3476us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:39571: BlockingRecv error: recv got EOF from 127.21.181.65:39571 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 12 07:10:42 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:42 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:42.830839 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:42.822758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:40539)
0812 07:10:42.823272 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:42.823287 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:42.823413 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:42.823724 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:42.823728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:42.824201 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:42.824367 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:42.824377 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.825552 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.825557 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:42.826448 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:42.826459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.827171 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.827178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:42.827325 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:42.827895 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:42.827912 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:42.830528 (+ 2616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":263,"threads_started":1}
W20260812 07:10:42.830971 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:42.822804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:40539 (local address 127.21.181.126:37881)
0812 07:10:42.822971 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:42.822977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:42.823364 (+ 387us) server_negotiation.cc:407] Connection header received
0812 07:10:42.823524 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:42.823529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:42.823593 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:42.823671 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:42.824508 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.825412 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.826592 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:42.827004 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:42.827110 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:42.830747 (+ 3637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:40539: BlockingRecv error: recv got EOF from 127.21.181.66:40539 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 12 07:10:43 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:43 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:43.266490 7185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:43.258468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:37275)
0812 07:10:43.259012 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:43.259028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:43.259183 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:43.259508 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:43.259513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:43.259942 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:43.260164 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:43.260178 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.261336 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.261340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:43.262246 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:43.262258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.262938 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.262943 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:43.263061 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:43.263610 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:43.263627 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:43.266242 (+ 2615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":266,"threads_started":1}
W20260812 07:10:43.266670 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:43.258569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:37275 (local address 127.21.181.126:37881)
0812 07:10:43.258770 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:43.258777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:43.259113 (+ 336us) server_negotiation.cc:407] Connection header received
0812 07:10:43.259307 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:43.259312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:43.259374 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:43.259452 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:43.260323 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.261188 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.262355 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.262801 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.262890 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:43.266469 (+ 3579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:37275: BlockingRecv error: recv got EOF from 127.21.181.67:37275 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 07:10:43 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:43 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:43.396926 7187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:43.389381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:37213)
0812 07:10:43.389882 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:43.389898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:43.390022 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:43.390279 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:43.390283 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:43.390679 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:43.390850 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:43.390858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.391771 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.391775 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:43.392728 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:43.392741 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.393343 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.393347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:43.393461 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:43.393984 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:43.394001 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:43.396685 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":263,"threads_started":1}
W20260812 07:10:43.397078 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:43.389454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:37213 (local address 127.21.181.126:37881)
0812 07:10:43.389704 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:43.389710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:43.389970 (+ 260us) server_negotiation.cc:407] Connection header received
0812 07:10:43.390112 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:43.390115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:43.390175 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:43.390252 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:43.390953 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.391672 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.392826 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.393234 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.393320 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:43.396857 (+ 3537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:37213: BlockingRecv error: recv got EOF from 127.21.181.65:37213 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 12 07:10:43 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:43 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:43.840236 7188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:43.832037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:35853)
0812 07:10:43.832591 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:43.832606 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:43.832735 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:43.833053 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:43.833058 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:43.833496 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:43.833670 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:43.833679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.834674 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.834678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:43.835611 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:43.835624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.836373 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.836380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:43.836541 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:43.837093 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:43.837112 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:43.839876 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":304,"threads_started":1}
W20260812 07:10:43.840478 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:43.832112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:35853 (local address 127.21.181.126:37881)
0812 07:10:43.832286 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:43.832292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:43.832689 (+ 397us) server_negotiation.cc:407] Connection header received
0812 07:10:43.832847 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:43.832852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:43.832920 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:43.833000 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:43.833816 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.834526 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.835751 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:43.836228 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:43.836336 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:43.840066 (+ 3730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:35853: BlockingRecv error: recv got EOF from 127.21.181.66:35853 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 07:10:44 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:44 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:44.275629 7189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:44.267360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:41959)
0812 07:10:44.267887 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:44.267906 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:44.268056 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:44.268462 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:44.268467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:44.268874 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:44.269058 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:44.269070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.270074 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.270080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:44.271051 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:44.271067 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.271777 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.271782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:44.271963 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:44.272574 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:44.272593 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:44.275383 (+ 2790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":304,"threads_started":1}
W20260812 07:10:44.275843 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:44.267430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:41959 (local address 127.21.181.126:37881)
0812 07:10:44.267621 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:44.267627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:44.268004 (+ 377us) server_negotiation.cc:407] Connection header received
0812 07:10:44.268210 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:44.268214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:44.268296 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:44.268414 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:44.269197 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.269938 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.271185 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.271631 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.271741 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:44.275602 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:41959: BlockingRecv error: recv got EOF from 127.21.181.67:41959 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 07:10:44 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:44 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:44.405426 7191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:44.397803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:55147)
0812 07:10:44.398306 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:44.398326 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:44.398451 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:44.398770 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:44.398774 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:44.399181 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:44.399347 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:44.399356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.400397 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.400401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:44.401305 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:44.401316 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.401932 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.401936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:44.402055 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:44.402569 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:44.402585 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:44.405192 (+ 2607us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":257,"threads_started":1}
W20260812 07:10:44.405618 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:44.397844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:55147 (local address 127.21.181.126:37881)
0812 07:10:44.398044 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:44.398050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:44.398397 (+ 347us) server_negotiation.cc:407] Connection header received
0812 07:10:44.398562 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:44.398568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:44.398633 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:44.398726 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:44.399449 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.400280 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.401408 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.401826 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.401912 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:44.405358 (+ 3446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:55147: BlockingRecv error: recv got EOF from 127.21.181.65:55147 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 12 07:10:44 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:44 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:44.849370 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:44.841197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:36703)
0812 07:10:44.841717 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:44.841734 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:44.841870 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:44.842227 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:44.842231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:44.842649 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:44.842798 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:44.842808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.844003 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.844008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:44.845056 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:44.845070 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.845746 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.845752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:44.845905 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:44.846443 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:44.846459 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:44.849143 (+ 2684us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":281,"threads_started":1}
W20260812 07:10:44.849558 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:44.841261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:36703 (local address 127.21.181.126:37881)
0812 07:10:44.841507 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:44.841514 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:44.841816 (+ 302us) server_negotiation.cc:407] Connection header received
0812 07:10:44.842004 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:44.842009 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:44.842098 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:44.842187 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:44.842948 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.843862 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.845202 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:44.845601 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:44.845709 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:44.849329 (+ 3620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:36703: BlockingRecv error: recv got EOF from 127.21.181.66:36703 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 12 07:10:45 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:45 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:45.284541 7194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:45.276603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:36343)
0812 07:10:45.277102 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:45.277119 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:45.277231 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:45.277568 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:45.277571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:45.277948 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:45.278101 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:45.278111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.279082 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.279085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:45.280015 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:45.280025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.280705 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.280709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:45.280824 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:45.281405 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:45.281423 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:45.284313 (+ 2890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":272,"threads_started":1}
W20260812 07:10:45.284858 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:45.276779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:36343 (local address 127.21.181.126:37881)
0812 07:10:45.276996 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:45.277002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:45.277194 (+ 192us) server_negotiation.cc:407] Connection header received
0812 07:10:45.277339 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:45.277345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:45.277423 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:45.277543 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:45.278253 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.278980 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.280185 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.280588 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.280678 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:45.284504 (+ 3826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:36343: BlockingRecv error: recv got EOF from 127.21.181.67:36343 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 12 07:10:45 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:45 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:45.414876 7195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:45.406304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:48859)
0812 07:10:45.406809 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:45.406828 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:45.406959 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:45.407238 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:45.407241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:45.407668 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:45.407889 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:45.407900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.408890 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.408895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:45.409879 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:45.409894 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.410816 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.410825 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:45.410993 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:45.411530 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:45.411548 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:45.414474 (+ 2926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":243,"threads_started":1}
W20260812 07:10:45.414964 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:45.406427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:48859 (local address 127.21.181.126:37881)
0812 07:10:45.406664 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:45.406670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:45.406906 (+ 236us) server_negotiation.cc:407] Connection header received
0812 07:10:45.407053 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:45.407056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:45.407117 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:45.407191 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:45.408022 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.408759 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.410045 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.410593 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.410731 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:45.414683 (+ 3952us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:48859: BlockingRecv error: recv got EOF from 127.21.181.65:48859 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 07:10:45 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:45 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:45.858139 7196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:45.850186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:41713)
0812 07:10:45.850697 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:45.850713 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:45.850844 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:45.851153 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:45.851158 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:45.851588 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:45.851774 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:45.851783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.852849 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.852853 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:45.853737 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:45.853749 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.854440 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.854447 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:45.854612 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:45.855164 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:45.855181 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:45.857804 (+ 2623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":286,"threads_started":1}
W20260812 07:10:45.858229 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:45.850281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:41713 (local address 127.21.181.126:37881)
0812 07:10:45.850485 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:45.850492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:45.850796 (+ 304us) server_negotiation.cc:407] Connection header received
0812 07:10:45.850954 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:45.850959 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:45.851023 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:45.851103 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:45.851918 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.852702 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.853876 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:45.854295 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:45.854403 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:45.858007 (+ 3604us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:41713: BlockingRecv error: recv got EOF from 127.21.181.66:41713 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:10:46 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:46 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:46.293207 7198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:46.285509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:47453)
0812 07:10:46.286086 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:46.286102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:46.286211 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:46.286536 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:46.286540 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:46.286929 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:46.287121 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:46.287130 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.288096 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.288100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:46.288972 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:46.288993 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.289584 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.289588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:46.289706 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:46.290235 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:46.290251 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:46.292891 (+ 2640us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":325,"threads_started":1}
W20260812 07:10:46.293361 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:46.285562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:47453 (local address 127.21.181.126:37881)
0812 07:10:46.285746 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:46.285753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:46.286175 (+ 422us) server_negotiation.cc:407] Connection header received
0812 07:10:46.286325 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:46.286330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:46.286410 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:46.286484 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:46.287242 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.287961 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.289086 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.289475 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.289563 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:46.293134 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:47453: BlockingRecv error: recv got EOF from 127.21.181.67:47453 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 12 07:10:46 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:46 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:46.423314 7199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:46.415864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:38889)
0812 07:10:46.416356 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:46.416372 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:46.416495 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:46.416754 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:46.416757 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:46.417126 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:46.417282 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:46.417291 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.418180 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.418183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:46.419097 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:46.419108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.419681 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.419685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:46.419808 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:46.420344 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:46.420361 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:46.423024 (+ 2663us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":293,"threads_started":1}
W20260812 07:10:46.423511 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:46.415919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:38889 (local address 127.21.181.126:37881)
0812 07:10:46.416175 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:46.416181 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:46.416445 (+ 264us) server_negotiation.cc:407] Connection header received
0812 07:10:46.416582 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:46.416586 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:46.416646 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:46.416722 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:46.417387 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.418073 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.419197 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.419581 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.419664 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:46.423209 (+ 3545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:38889: BlockingRecv error: recv got EOF from 127.21.181.65:38889 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 12 07:10:46 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:46 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:46.869424 7202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:46.859107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:55609)
0812 07:10:46.859621 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:46.859636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:46.859772 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:46.860158 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:46.860163 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:46.860705 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:46.860867 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:46.860877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.861854 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.861857 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:46.862759 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:46.862770 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.863384 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.863388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:46.863515 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:46.866558 (+ 3043us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:46.866575 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:46.869188 (+ 2613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":273,"threads_started":1}
W20260812 07:10:46.869578 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:46.859162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:55609 (local address 127.21.181.126:37881)
0812 07:10:46.859358 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:46.859365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:46.859697 (+ 332us) server_negotiation.cc:407] Connection header received
0812 07:10:46.859893 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:46.859899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:46.859979 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:46.860103 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:46.860996 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.861755 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.862862 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:46.863285 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:46.863370 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:46.869370 (+ 6000us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:55609: BlockingRecv error: recv got EOF from 127.21.181.66:55609 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 12 07:10:47 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:47 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:47.302623 7203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:47.294122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:34565)
0812 07:10:47.294631 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:47.294647 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:47.294777 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:47.295121 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:47.295127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:47.295593 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:47.295753 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:47.295764 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.296825 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.296829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:47.297892 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:47.297906 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.298621 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.298627 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:47.298793 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:47.299419 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:47.299443 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:47.302300 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":290,"threads_started":1}
W20260812 07:10:47.302742 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:47.294162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:34565 (local address 127.21.181.126:37881)
0812 07:10:47.294339 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:47.294345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:47.294730 (+ 385us) server_negotiation.cc:407] Connection header received
0812 07:10:47.294895 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:47.294901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:47.294970 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:47.295062 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:47.295890 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.296690 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.298024 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.298459 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.298576 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:47.302522 (+ 3946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:34565: BlockingRecv error: recv got EOF from 127.21.181.67:34565 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 12 07:10:47 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:47 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:47.432461 7204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:47.424263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:34065)
0812 07:10:47.424717 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:47.424732 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:47.424848 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:47.425114 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:47.425117 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:47.425512 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:47.425677 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:47.425686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.426575 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.426578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:47.427720 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:47.427738 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.428708 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.428713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:47.428848 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:47.429376 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:47.429392 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:47.432135 (+ 2743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":229,"threads_started":1}
W20260812 07:10:47.432559 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:47.424335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:34065 (local address 127.21.181.126:37881)
0812 07:10:47.424574 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:47.424580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:47.424803 (+ 223us) server_negotiation.cc:407] Connection header received
0812 07:10:47.424935 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:47.424938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:47.425017 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:47.425094 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:47.425778 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.426469 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.427864 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.428545 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.428659 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:47.432328 (+ 3669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:34065: BlockingRecv error: recv got EOF from 127.21.181.65:34065 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 12 07:10:47 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:47 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:47.878594 7205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:47.870343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:45961)
0812 07:10:47.870844 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:47.870859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:47.871018 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:47.871332 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:47.871337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:47.871741 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:47.871904 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:47.871913 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.872990 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.872996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:47.873908 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:47.873920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.874651 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.874658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:47.874819 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:47.875357 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:47.875373 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:47.878359 (+ 2986us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":245,"threads_started":1}
W20260812 07:10:47.878731 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:47.870504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:45961 (local address 127.21.181.126:37881)
0812 07:10:47.870698 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:47.870704 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:47.870934 (+ 230us) server_negotiation.cc:407] Connection header received
0812 07:10:47.871138 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:47.871142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:47.871203 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:47.871279 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:47.872050 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.872836 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.874048 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:47.874480 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:47.874594 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:47.878547 (+ 3953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:45961: BlockingRecv error: recv got EOF from 127.21.181.66:45961 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 12 07:10:48 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:48 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:48.316505 7206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:48.303559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:55783)
0812 07:10:48.304127 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:48.304144 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:48.304259 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:48.304584 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:48.304588 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:48.305086 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:48.305275 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:48.305286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.306341 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.306346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:48.307273 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:48.307286 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.307919 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.307923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:48.308053 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:48.313296 (+ 5243us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:48.313315 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:48.316205 (+ 2890us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":278,"threads_started":1}
W20260812 07:10:48.316685 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:48.303647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:55783 (local address 127.21.181.126:37881)
0812 07:10:48.303835 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:48.303841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:48.304224 (+ 383us) server_negotiation.cc:407] Connection header received
0812 07:10:48.304369 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:48.304374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:48.304441 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:48.304546 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:48.305399 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.306208 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.307383 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.307788 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.307874 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:48.316427 (+ 8553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:55783: BlockingRecv error: recv got EOF from 127.21.181.67:55783 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 12 07:10:48 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:48 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:48.440825 7208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:48.433344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:52959)
0812 07:10:48.433862 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:48.433878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:48.433993 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:48.434250 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:48.434254 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:48.434648 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:48.434828 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:48.434839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.435776 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.435779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:48.436751 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:48.436764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.437391 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.437395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:48.437519 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:48.438033 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:48.438050 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:48.440600 (+ 2550us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":279,"threads_started":1}
W20260812 07:10:48.440948 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:48.433411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:52959 (local address 127.21.181.126:37881)
0812 07:10:48.433717 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:48.433723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:48.433949 (+ 226us) server_negotiation.cc:407] Connection header received
0812 07:10:48.434087 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:48.434091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:48.434152 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:48.434220 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:48.434948 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.435678 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.436854 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.437286 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.437374 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:48.440759 (+ 3385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:52959: BlockingRecv error: recv got EOF from 127.21.181.65:52959 (error 108)
Metrics: {"server-negotiator.queue_time_us":152}
Aug 12 07:10:48 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:48 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:48.887341 7209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:48.879422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:54369)
0812 07:10:48.879874 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:48.879890 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:48.880015 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:48.880315 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:48.880319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:48.880685 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:48.880862 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:48.880870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.882041 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.882047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:48.883184 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:48.883198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.883846 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.883849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:48.883959 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:48.884528 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:48.884546 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:48.887103 (+ 2557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":244,"threads_started":1}
W20260812 07:10:48.887488 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:48.879501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:54369 (local address 127.21.181.126:37881)
0812 07:10:48.879703 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:48.879710 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:48.879963 (+ 253us) server_negotiation.cc:407] Connection header received
0812 07:10:48.880140 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:48.880143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:48.880208 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:48.880282 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:48.880965 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.881890 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.883304 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:48.883740 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:48.883834 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:48.887286 (+ 3452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:54369: BlockingRecv error: recv got EOF from 127.21.181.66:54369 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 12 07:10:49 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:49 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.67@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:49.325464 7210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:49.317455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.67:57653)
0812 07:10:49.317953 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:49.317968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:49.318099 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:49.318512 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:49.318518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:49.319020 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:49.319220 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:49.319230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.320259 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.320265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:49.321172 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:49.321184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.321864 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.321870 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:49.322045 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:49.322605 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:49.322622 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:49.325228 (+ 2606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":253,"threads_started":1}
W20260812 07:10:49.325639 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:49.317530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57653 (local address 127.21.181.126:37881)
0812 07:10:49.317677 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:49.317682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:49.318073 (+ 391us) server_negotiation.cc:407] Connection header received
0812 07:10:49.318254 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:49.318259 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:49.318342 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:49.318434 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:49.319383 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.320118 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.321314 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.321728 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.321836 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:49.325407 (+ 3571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.67:57653: BlockingRecv error: recv got EOF from 127.21.181.67:57653 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 12 07:10:49 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:49 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.65@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:49.449662 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:49.441902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.65:53563)
0812 07:10:49.442380 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:49.442396 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:49.442517 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:49.442784 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:49.442788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:49.443174 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:49.443343 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:49.443352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.444308 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.444312 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:49.445266 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:49.445282 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.446151 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.446159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:49.446291 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:49.446863 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:49.446880 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:49.449399 (+ 2519us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":222,"threads_started":1}
W20260812 07:10:49.449774 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:49.441985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:53563 (local address 127.21.181.126:37881)
0812 07:10:49.442213 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:49.442219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:49.442467 (+ 248us) server_negotiation.cc:407] Connection header received
0812 07:10:49.442606 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:49.442609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:49.442669 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:49.442745 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:49.443445 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.444203 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.445403 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.445970 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.446075 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:49.449572 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.65:53563: BlockingRecv error: recv got EOF from 127.21.181.65:53563 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 12 07:10:49 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
Aug 12 07:10:49 dist-test-slave-lgr5 krb5kdc[5292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.21.181.66@KRBTEST.COM for oryx/127.21.181.126@KRBTEST.COM, Server not found in Kerberos database
I20260812 07:10:49.896164 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:49.888348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:37881 (local address 127.21.181.66:50825)
0812 07:10:49.888797 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:49.888813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:49.888922 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:49.889194 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:49.889197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:49.889582 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:49.889745 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:49.889753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.890664 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.890667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:49.891565 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:49.891576 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.892377 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.892384 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:49.892558 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:49.893153 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:49.893177 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:49.895883 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.21.181.126:37881: Server oryx/127.21.181.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":244,"threads_started":1}
W20260812 07:10:49.896315 6691 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:49.888431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:50825 (local address 127.21.181.126:37881)
0812 07:10:49.888625 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:49.888632 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:49.888882 (+ 250us) server_negotiation.cc:407] Connection header received
0812 07:10:49.889032 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:49.889035 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:49.889096 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:49.889166 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:49.889845 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.890559 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.891674 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:49.892228 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:49.892348 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:49.896103 (+ 3755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.21.181.66:50825: BlockingRecv error: recv got EOF from 127.21.181.66:50825 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:10:50.178781 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6397
I20260812 07:10:50.189765 7218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:50.179665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48976 (local address 127.21.181.65:42451)
0812 07:10:50.180204 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:50.180212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:50.180238 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:10:50.180294 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:50.180301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:50.180515 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:50.180658 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:50.181984 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.182914 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.184034 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.184690 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.184790 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:50.187061 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:50.187070 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:50.187084 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:50.187120 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:50.189296 (+ 2176us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.181.65@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 07:10:50.189545 (+ 249us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48976: Request ticket server kudu/127.21.181.65@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":216,"threads_started":1}
W20260812 07:10:50.189872 7218 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48976: Request ticket server kudu/127.21.181.65@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 07:10:50.189945 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6397: Not authorized: Client connection negotiation failed: client connection to 127.21.181.65:42451: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.181.65@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 07:10:50.190140 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 6397
I20260812 07:10:50.229871 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6689
I20260812 07:10:50.241041 7219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:50.230744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58504 (local address 127.21.181.66:40579)
0812 07:10:50.231318 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:50.231325 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:50.231355 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:10:50.231415 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:50.231422 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:50.231638 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:50.231788 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:50.233045 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.234137 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.235334 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.235870 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.235964 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:50.238258 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:50.238268 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:50.238281 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:50.238317 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:50.240559 (+ 2242us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.181.66@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 07:10:50.240800 (+ 241us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58504: Request ticket server kudu/127.21.181.66@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":257,"threads_started":1}
W20260812 07:10:50.241145 7219 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58504: Request ticket server kudu/127.21.181.66@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 07:10:50.241132 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6689: Not authorized: Client connection negotiation failed: client connection to 127.21.181.66:40579: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.181.66@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 07:10:50.241289 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 6689
I20260812 07:10:50.271497 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6907
I20260812 07:10:50.282534 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:50.272393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49600 (local address 127.21.181.67:39417)
0812 07:10:50.272974 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:50.272983 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:50.273011 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:10:50.273081 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:50.273087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:50.273301 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:50.273459 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:50.274657 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.275704 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.276864 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.277422 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.277533 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:50.279770 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:50.279778 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:50.279793 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:50.279820 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:50.282036 (+ 2216us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.181.67@KRBTEST.COM not found in keytab (ticket kvno 2)
0812 07:10:50.282287 (+ 251us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49600: Request ticket server kudu/127.21.181.67@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":294,"threads_started":1}
W20260812 07:10:50.282644 7220 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49600: Request ticket server kudu/127.21.181.67@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260812 07:10:50.282624 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6907: Not authorized: Client connection negotiation failed: client connection to 127.21.181.67:39417: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.21.181.67@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260812 07:10:50.282773 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 6907
I20260812 07:10:50.306722 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 6271
I20260812 07:10:50.307911 6388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:50.425374 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 6271
2026-08-12T07:10:50Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21204 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-12T07:10:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:50Z Disabled control of system clock
I20260812 07:10:50.509936 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34329
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:38883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:34329
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260812 07:10:50.822540 7228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:50.822821 7228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:50.822924 7228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:50.829953 7228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:50.830044 7228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:50.830111 7228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:50.830186 7228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:50.830241 7228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:50.839286 7228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:34329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:50.840596 7228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:50.842285 7228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:50.855427 7240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:50.855453 7228 server_base.cc:1034] running on GCE node
W20260812 07:10:50.855484 7247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:50.855510 7239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:50.856647 7228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:50.857959 7228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:50.859241 7228 hybrid_clock.cc:648] HybridClock initialized: now 1786518650859190 us; error 44 us; skew 500 ppm
I20260812 07:10:50.859580 7228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:50.861706 7228 webserver.cc:460] Webserver started at http://127.21.181.126:44649/ using document root <none> and password file <none>
I20260812 07:10:50.862370 7228 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:50.862474 7228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:50.862777 7228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:50.865585 7228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "6188b232d45840a6988be59bdcd0b5f6"
format_stamp: "Formatted at 2026-08-12 07:10:50 on dist-test-slave-lgr5"
server_key: "4b4ea79c16d5783ba03a712d0a0814ac"
server_key_iv: "91f38867b41c999736b66c3181cd9109"
server_key_version: "encryptionkey@0"
I20260812 07:10:50.866338 7228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "6188b232d45840a6988be59bdcd0b5f6"
format_stamp: "Formatted at 2026-08-12 07:10:50 on dist-test-slave-lgr5"
server_key: "4b4ea79c16d5783ba03a712d0a0814ac"
server_key_iv: "91f38867b41c999736b66c3181cd9109"
server_key_version: "encryptionkey@0"
I20260812 07:10:50.871078 7228 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:10:50.874481 7258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:50.875806 7228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:10:50.875973 7228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "6188b232d45840a6988be59bdcd0b5f6"
format_stamp: "Formatted at 2026-08-12 07:10:50 on dist-test-slave-lgr5"
server_key: "4b4ea79c16d5783ba03a712d0a0814ac"
server_key_iv: "91f38867b41c999736b66c3181cd9109"
server_key_version: "encryptionkey@0"
I20260812 07:10:50.876225 7228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:50.898098 7228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:50.898847 7228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:50.899098 7228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:50.915688 7228 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:34329
I20260812 07:10:50.915709 7345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:34329 every 8 connection(s)
I20260812 07:10:50.916975 7228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:50.917469 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 7228
I20260812 07:10:50.917701 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:50.918561 22229 external_mini_cluster.cc:1428] Setting key 61648db63cff52118a105b0720223e86
I20260812 07:10:50.921428 7346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:50.931955 7353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:50.920607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44928 (local address 127.21.181.126:34329)
0812 07:10:50.921558 (+ 951us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:50.921571 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:50.921618 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:10:50.923212 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:50.923234 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:50.923644 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:50.923970 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:50.924615 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.925904 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.927664 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:50.928349 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:50.928477 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:50.929655 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:50.929662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:50.929673 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:50.930200 (+ 527us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:50.930283 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:50.930307 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:50.930398 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:50.930654 (+ 256us) server_negotiation.cc:299] Negotiation successful
0812 07:10:50.930955 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":348,"threads_started":1}
I20260812 07:10:50.933292 7346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6: Bootstrap starting.
I20260812 07:10:50.936249 7346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:50.937170 7346 log.cc:824] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:50.939692 7346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6: No bootstrap required, opened a new log
I20260812 07:10:50.946401 7346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6188b232d45840a6988be59bdcd0b5f6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34329 } }
I20260812 07:10:50.946703 7346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:50.946800 7346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6188b232d45840a6988be59bdcd0b5f6, State: Initialized, Role: FOLLOWER
I20260812 07:10:50.947325 7346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6188b232d45840a6988be59bdcd0b5f6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34329 } }
I20260812 07:10:50.947535 7346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:50.947654 7346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:50.947808 7346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:50.950040 7346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6188b232d45840a6988be59bdcd0b5f6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34329 } }
I20260812 07:10:50.950465 7346 leader_election.cc:302] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6188b232d45840a6988be59bdcd0b5f6; no voters:
I20260812 07:10:50.950966 7346 leader_election.cc:288] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:50.951089 7356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:50.951751 7356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [term 1 LEADER]: Becoming Leader. State: Replica: 6188b232d45840a6988be59bdcd0b5f6, State: Running, Role: LEADER
I20260812 07:10:50.952358 7356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6188b232d45840a6988be59bdcd0b5f6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34329 } }
I20260812 07:10:50.952942 7346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:50.954789 7359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6188b232d45840a6988be59bdcd0b5f6. Latest consensus state: current_term: 1 leader_uuid: "6188b232d45840a6988be59bdcd0b5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6188b232d45840a6988be59bdcd0b5f6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34329 } } }
I20260812 07:10:50.955284 7359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:50.960289 7358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6188b232d45840a6988be59bdcd0b5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6188b232d45840a6988be59bdcd0b5f6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34329 } } }
I20260812 07:10:50.960600 7358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6 [sys.catalog]: This master's current role is: LEADER
W20260812 07:10:50.962622 7387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:10:50.962723 7387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:10:50.962841 7371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:50.964954 7371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:50.973315 7371 catalog_manager.cc:1347] Generated new cluster ID: 087a3e19c4ec469381a683d35b8d2ce0
I20260812 07:10:50.973429 7371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:50.983714 7371 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:50.985391 7371 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:51.005695 7371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6188b232d45840a6988be59bdcd0b5f6: Generated new TSK 0
I20260812 07:10:51.006703 7371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:10:51.014214 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34329
--builtin_ntp_servers=127.21.181.84:38883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260812 07:10:51.342387 7392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:51.342682 7392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:51.342839 7392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:51.350996 7392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:51.351105 7392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:51.351251 7392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:51.360435 7392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:51.361642 7392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:51.363477 7392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:51.376042 7398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:51.376514 7407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:51.376585 7401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:51.377238 7392 server_base.cc:1034] running on GCE node
I20260812 07:10:51.377784 7392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:51.378677 7392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:51.379863 7392 hybrid_clock.cc:648] HybridClock initialized: now 1786518651379814 us; error 41 us; skew 500 ppm
I20260812 07:10:51.380288 7392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:51.382476 7392 webserver.cc:460] Webserver started at http://127.21.181.65:34009/ using document root <none> and password file <none>
I20260812 07:10:51.383106 7392 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:51.383191 7392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:51.383463 7392 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:51.386169 7392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "8906c4561f914513921f92f0a9671481"
format_stamp: "Formatted at 2026-08-12 07:10:51 on dist-test-slave-lgr5"
server_key: "cc526c65cb71f1d86b9c68f4a5425448"
server_key_iv: "7f2c76714d48b6e2c6dde3d7227cf686"
server_key_version: "encryptionkey@0"
I20260812 07:10:51.386886 7392 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "8906c4561f914513921f92f0a9671481"
format_stamp: "Formatted at 2026-08-12 07:10:51 on dist-test-slave-lgr5"
server_key: "cc526c65cb71f1d86b9c68f4a5425448"
server_key_iv: "7f2c76714d48b6e2c6dde3d7227cf686"
server_key_version: "encryptionkey@0"
I20260812 07:10:51.391562 7392 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:10:51.394711 7417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:51.396029 7392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:10:51.396214 7392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "8906c4561f914513921f92f0a9671481"
format_stamp: "Formatted at 2026-08-12 07:10:51 on dist-test-slave-lgr5"
server_key: "cc526c65cb71f1d86b9c68f4a5425448"
server_key_iv: "7f2c76714d48b6e2c6dde3d7227cf686"
server_key_version: "encryptionkey@0"
I20260812 07:10:51.396399 7392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:51.413558 7392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:51.414276 7392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:51.414548 7392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:51.415442 7392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:51.416797 7392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:51.416885 7392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:51.416993 7392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:51.417073 7392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:51.445324 7392 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:36101
I20260812 07:10:51.445367 7627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:36101 every 8 connection(s)
I20260812 07:10:51.446563 7392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:51.450727 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 7392
I20260812 07:10:51.450940 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:51.451719 22229 external_mini_cluster.cc:1428] Setting key e678464fe15bdbf241b642de8f687e62
I20260812 07:10:51.454850 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34329
--builtin_ntp_servers=127.21.181.84:38883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 07:10:51.460242 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:51.448903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:40121 (local address 127.21.181.126:34329)
0812 07:10:51.449304 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:51.449309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:51.450540 (+ 1231us) server_negotiation.cc:407] Connection header received
0812 07:10:51.453059 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:51.453063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:51.453125 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:51.453216 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:51.454602 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.455381 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.457533 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.457962 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.458077 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:51.459491 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:51.459498 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:51.459501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:51.459519 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:51.459565 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:51.459578 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:51.459645 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:51.459859 (+ 214us) server_negotiation.cc:299] Negotiation successful
0812 07:10:51.459932 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":201,"threads_started":1}
I20260812 07:10:51.461620 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:51.449427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34329 (local address 127.21.181.65:40121)
0812 07:10:51.450371 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:51.450420 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:51.452768 (+ 2348us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:51.453504 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:51.453515 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:51.453627 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:51.454423 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:51.454451 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.455530 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.455535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:51.457385 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:51.457407 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.458183 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.458189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:51.458446 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:51.459129 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:51.459133 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:51.459178 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:51.459261 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:51.459674 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:51.459683 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:51.459690 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:10:51.459852 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 07:10:51.460240 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":264,"threads_started":1}
I20260812 07:10:51.462980 7628 heartbeater.cc:347] Connected to a master server at 127.21.181.126:34329
I20260812 07:10:51.463238 7628 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:51.463946 7628 heartbeater.cc:511] Master 127.21.181.126:34329 requested a full tablet report, sending...
I20260812 07:10:51.465869 7285 ts_manager.cc:194] Registered new tserver with Master: 8906c4561f914513921f92f0a9671481 (127.21.181.65:36101)
I20260812 07:10:51.467744 7285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.65:40121
W20260812 07:10:51.772168 7634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:51.772428 7634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:51.772553 7634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:51.779475 7634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:51.779568 7634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:51.779713 7634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:51.788391 7634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:51.789532 7634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:51.791189 7634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:51.803493 7647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:51.803570 7653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:51.803599 7649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:51.804388 7634 server_base.cc:1034] running on GCE node
I20260812 07:10:51.804924 7634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:51.805733 7634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:51.806926 7634 hybrid_clock.cc:648] HybridClock initialized: now 1786518651806878 us; error 41 us; skew 500 ppm
I20260812 07:10:51.807224 7634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:51.809279 7634 webserver.cc:460] Webserver started at http://127.21.181.66:40359/ using document root <none> and password file <none>
I20260812 07:10:51.809899 7634 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:51.809993 7634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:51.810292 7634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:51.812954 7634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "c8d118813a1f4e0dbc7f31a0dbc7b3ea"
format_stamp: "Formatted at 2026-08-12 07:10:51 on dist-test-slave-lgr5"
server_key: "b13863d230d6f813e9baf0d8c0d14693"
server_key_iv: "e25c3c58c167c146cc2b6549c61e1f86"
server_key_version: "encryptionkey@0"
I20260812 07:10:51.813706 7634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "c8d118813a1f4e0dbc7f31a0dbc7b3ea"
format_stamp: "Formatted at 2026-08-12 07:10:51 on dist-test-slave-lgr5"
server_key: "b13863d230d6f813e9baf0d8c0d14693"
server_key_iv: "e25c3c58c167c146cc2b6549c61e1f86"
server_key_version: "encryptionkey@0"
I20260812 07:10:51.818382 7634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:10:51.821497 7669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:51.822693 7634 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:10:51.822847 7634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "c8d118813a1f4e0dbc7f31a0dbc7b3ea"
format_stamp: "Formatted at 2026-08-12 07:10:51 on dist-test-slave-lgr5"
server_key: "b13863d230d6f813e9baf0d8c0d14693"
server_key_iv: "e25c3c58c167c146cc2b6549c61e1f86"
server_key_version: "encryptionkey@0"
I20260812 07:10:51.823038 7634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:51.838043 7634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:51.838742 7634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:51.839001 7634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:51.839829 7634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:51.841187 7634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:51.841282 7634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:51.841389 7634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:51.841467 7634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:51.871367 7634 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:45667
I20260812 07:10:51.871415 7844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:45667 every 8 connection(s)
I20260812 07:10:51.872575 7634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:51.881208 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 7634
I20260812 07:10:51.881366 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:51.881783 22229 external_mini_cluster.cc:1428] Setting key 9b1249f81afcd239c390daf2eafb6cb9
I20260812 07:10:51.884353 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34329
--builtin_ntp_servers=127.21.181.84:38883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260812 07:10:51.886859 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:51.874925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:45755 (local address 127.21.181.126:34329)
0812 07:10:51.875150 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:51.875157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:51.876431 (+ 1274us) server_negotiation.cc:407] Connection header received
0812 07:10:51.878078 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:51.878083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:51.878147 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:51.878231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:51.879203 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.879928 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.881593 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.882031 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.882144 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:51.886065 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:51.886070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:51.886073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:51.886089 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:51.886122 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:51.886132 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:51.886206 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:51.886443 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 07:10:51.886540 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:10:51.887760 7847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:51.875420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34329 (local address 127.21.181.66:45755)
0812 07:10:51.876292 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:51.876325 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:51.877835 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:51.878416 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:51.878423 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:51.878512 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:51.879041 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:51.879082 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.880070 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.880074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:51.881426 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:51.881442 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:51.882255 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:51.882263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:51.882508 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:51.885764 (+ 3256us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:51.885766 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:51.885802 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:51.885871 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:51.886226 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:51.886238 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:51.886246 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:10:51.886392 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 07:10:51.886757 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":256,"threads_started":1}
I20260812 07:10:51.889118 7845 heartbeater.cc:347] Connected to a master server at 127.21.181.126:34329
I20260812 07:10:51.889384 7845 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:51.890085 7845 heartbeater.cc:511] Master 127.21.181.126:34329 requested a full tablet report, sending...
I20260812 07:10:51.891414 7285 ts_manager.cc:194] Registered new tserver with Master: c8d118813a1f4e0dbc7f31a0dbc7b3ea (127.21.181.66:45667)
I20260812 07:10:51.892403 7285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.66:45755
W20260812 07:10:52.206185 7850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:52.206425 7850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:52.206512 7850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:52.213439 7850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:52.213522 7850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:52.213625 7850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:52.222139 7850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:38883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:34329
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:52.223206 7850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:52.224855 7850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:52.237138 7855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:52.237176 7861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:52.237223 7857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:52.238355 7850 server_base.cc:1034] running on GCE node
I20260812 07:10:52.238911 7850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:52.239748 7850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:52.240942 7850 hybrid_clock.cc:648] HybridClock initialized: now 1786518652240895 us; error 36 us; skew 500 ppm
I20260812 07:10:52.241240 7850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:52.243253 7850 webserver.cc:460] Webserver started at http://127.21.181.67:42011/ using document root <none> and password file <none>
I20260812 07:10:52.243871 7850 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:52.243969 7850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:52.244314 7850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:52.246923 7850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "218ba33d993b49cdbe6df48c9b2288ee"
format_stamp: "Formatted at 2026-08-12 07:10:52 on dist-test-slave-lgr5"
server_key: "c142ea6e141b0ed17b65ad8cae55c11e"
server_key_iv: "21760923034e467f0d8f7a6a65820049"
server_key_version: "encryptionkey@0"
I20260812 07:10:52.247694 7850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "218ba33d993b49cdbe6df48c9b2288ee"
format_stamp: "Formatted at 2026-08-12 07:10:52 on dist-test-slave-lgr5"
server_key: "c142ea6e141b0ed17b65ad8cae55c11e"
server_key_iv: "21760923034e467f0d8f7a6a65820049"
server_key_version: "encryptionkey@0"
I20260812 07:10:52.252384 7850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:10:52.255499 7868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:52.256726 7850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:52.256867 7850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "218ba33d993b49cdbe6df48c9b2288ee"
format_stamp: "Formatted at 2026-08-12 07:10:52 on dist-test-slave-lgr5"
server_key: "c142ea6e141b0ed17b65ad8cae55c11e"
server_key_iv: "21760923034e467f0d8f7a6a65820049"
server_key_version: "encryptionkey@0"
I20260812 07:10:52.257066 7850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:52.279229 7850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:52.279930 7850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:52.280241 7850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:52.281101 7850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:52.282392 7850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:52.282474 7850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:52.282548 7850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:52.282599 7850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:52.312186 7850 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:40983
I20260812 07:10:52.312260 8065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:40983 every 8 connection(s)
I20260812 07:10:52.313352 7850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:52.319846 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 7850
I20260812 07:10:52.320007 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:52.320448 22229 external_mini_cluster.cc:1428] Setting key eb68c0443e3124fb514f87a6847feb34
I20260812 07:10:52.325096 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:52.315711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:39789 (local address 127.21.181.126:34329)
0812 07:10:52.315884 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:52.315888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:52.317183 (+ 1295us) server_negotiation.cc:407] Connection header received
0812 07:10:52.318882 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:52.318888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:52.318963 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:52.319062 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:52.320150 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.320952 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.322500 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.323102 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.323229 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:52.324484 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:52.324488 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:52.324492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:52.324507 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:52.324535 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:52.324547 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:52.324609 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:52.324773 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 07:10:52.324881 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:10:52.325963 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:52.316174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:34329 (local address 127.21.181.67:39789)
0812 07:10:52.317026 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:52.317068 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:52.318613 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:52.319294 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:52.319302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:52.319385 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:52.319943 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:52.319962 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.321184 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.321190 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:52.322365 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:52.322378 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.323299 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.323307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:52.323584 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:52.324216 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:52.324219 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:52.324254 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:52.324326 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:52.324618 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:52.324625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:52.324630 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:10:52.324757 (+ 127us) client_negotiation.cc:240] Negotiation successful
0812 07:10:52.325024 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":282,"threads_started":1}
I20260812 07:10:52.327148 8069 heartbeater.cc:347] Connected to a master server at 127.21.181.126:34329
I20260812 07:10:52.327440 8069 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:52.328123 8069 heartbeater.cc:511] Master 127.21.181.126:34329 requested a full tablet report, sending...
I20260812 07:10:52.329427 7285 ts_manager.cc:194] Registered new tserver with Master: 218ba33d993b49cdbe6df48c9b2288ee (127.21.181.67:40983)
I20260812 07:10:52.330155 7285 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.67:39789
I20260812 07:10:52.335516 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:10:52.341210 7630 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:52.339796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44934 (local address 127.21.181.126:34329)
0812 07:10:52.339979 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:52.339986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:52.340400 (+ 414us) server_negotiation.cc:407] Connection header received
0812 07:10:52.340534 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:52.340538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:52.340603 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:52.340675 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:52.341010 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44934: BlockingRecv error: recv got EOF from 127.0.0.1:44934 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:10:52.342095 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 7392
I20260812 07:10:52.348965 8082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:52.342944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35660 (local address 127.21.181.65:36101)
0812 07:10:52.343451 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:52.343457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:52.343481 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:10:52.343543 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:52.343549 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:52.343704 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:52.343817 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:52.344182 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.345169 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.346291 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.346787 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.346877 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:52.347756 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:52.347763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:52.347773 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:52.348331 (+ 558us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:52.348375 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:52.348395 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:52.348477 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:52.348590 (+ 113us) server_negotiation.cc:299] Negotiation successful
0812 07:10:52.348700 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":269,"threads_started":1}
I20260812 07:10:52.350342 7619 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:52.430999 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 7392
I20260812 07:10:52.451720 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 7634
I20260812 07:10:52.459052 8085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:52.452587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36580 (local address 127.21.181.66:45667)
0812 07:10:52.453103 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:52.453110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:52.453137 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:10:52.453215 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:52.453221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:52.453426 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:52.453608 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:52.453949 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.454932 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.456055 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.456730 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.456823 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:52.457750 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:52.457758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:52.457772 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:52.458377 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:52.458429 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:52.458450 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:52.458544 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:52.458647 (+ 103us) server_negotiation.cc:299] Negotiation successful
0812 07:10:52.458765 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":261,"threads_started":1}
I20260812 07:10:52.460292 7839 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:52.540320 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 7634
I20260812 07:10:52.567698 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 7850
I20260812 07:10:52.576160 8088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:52.568611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.21.181.67:40983)
0812 07:10:52.569237 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:52.569244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:52.569273 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:10:52.569332 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:52.569338 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:52.569559 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:52.569701 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:52.570182 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.571640 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.573018 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:52.573595 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:52.573709 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:52.574705 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:52.574715 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:52.574728 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:52.575312 (+ 584us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:52.575358 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:52.575379 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:52.575478 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:52.575667 (+ 189us) server_negotiation.cc:299] Negotiation successful
0812 07:10:52.575796 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":313,"threads_started":1}
I20260812 07:10:52.577363 8056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:52.656919 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 7850
I20260812 07:10:52.679292 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 7228
I20260812 07:10:52.680477 7333 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:52.780279 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 7228
2026-08-12T07:10:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2349 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-12T07:10:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:52Z Disabled control of system clock
I20260812 07:10:52.859746 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45305
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:34041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:45305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 07:10:53.176460 8101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:53.176733 8101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:53.176822 8101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:53.183840 8101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:53.183935 8101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:53.183979 8101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:53.184033 8101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:53.184094 8101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:53.192852 8101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:45305
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:53.194007 8101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:53.195701 8101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:53.206929 8116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:53.207010 8108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:53.207018 8110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:53.207767 8101 server_base.cc:1034] running on GCE node
I20260812 07:10:53.208514 8101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:53.209794 8101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:53.211005 8101 hybrid_clock.cc:648] HybridClock initialized: now 1786518653210949 us; error 41 us; skew 500 ppm
I20260812 07:10:53.211321 8101 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:53.213497 8101 webserver.cc:460] Webserver started at http://127.21.181.126:42429/ using document root <none> and password file <none>
I20260812 07:10:53.214147 8101 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:53.214248 8101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:53.214563 8101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:53.217427 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "22f9182024ca449898128e8c395f8609"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-lgr5"
server_key: "8ea3a2d90d6f937ec6ffea91a37e62fb"
server_key_iv: "e4cbcc833102987ae515952e381deadb"
server_key_version: "encryptionkey@0"
I20260812 07:10:53.218197 8101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "22f9182024ca449898128e8c395f8609"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-lgr5"
server_key: "8ea3a2d90d6f937ec6ffea91a37e62fb"
server_key_iv: "e4cbcc833102987ae515952e381deadb"
server_key_version: "encryptionkey@0"
I20260812 07:10:53.222961 8101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:10:53.226133 8125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:53.227322 8101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:10:53.227486 8101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "22f9182024ca449898128e8c395f8609"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-lgr5"
server_key: "8ea3a2d90d6f937ec6ffea91a37e62fb"
server_key_iv: "e4cbcc833102987ae515952e381deadb"
server_key_version: "encryptionkey@0"
I20260812 07:10:53.227684 8101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:53.233136 8101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:53.233850 8101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:53.234138 8101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:53.250836 8101 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:45305
I20260812 07:10:53.250866 8209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:45305 every 8 connection(s)
I20260812 07:10:53.252041 8101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:53.255496 8210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:53.259994 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 8101
I20260812 07:10:53.260213 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:53.261080 22229 external_mini_cluster.cc:1428] Setting key a48988f32745b954ecd5c0bb895448d1
I20260812 07:10:53.266520 8210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609: Bootstrap starting.
I20260812 07:10:53.268996 8216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:53.263100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49730 (local address 127.21.181.126:45305)
0812 07:10:53.264062 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:53.264072 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:53.264163 (+ 91us) server_negotiation.cc:407] Connection header received
0812 07:10:53.265759 (+ 1596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:53.265781 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:53.266159 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:53.266457 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:53.267512 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:53.267518 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:53.267530 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:53.267544 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:53.267599 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:53.267614 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:53.267777 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:53.267893 (+ 116us) server_negotiation.cc:299] Negotiation successful
0812 07:10:53.268179 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":358,"threads_started":1}
I20260812 07:10:53.269783 8210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:53.270750 8210 log.cc:824] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:53.273314 8210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609: No bootstrap required, opened a new log
I20260812 07:10:53.280052 8210 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f9182024ca449898128e8c395f8609" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45305 } }
I20260812 07:10:53.280423 8210 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:53.280509 8210 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22f9182024ca449898128e8c395f8609, State: Initialized, Role: FOLLOWER
I20260812 07:10:53.281055 8210 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f9182024ca449898128e8c395f8609" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45305 } }
I20260812 07:10:53.281334 8210 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:53.281472 8210 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:53.281651 8210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:53.283595 8210 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f9182024ca449898128e8c395f8609" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45305 } }
I20260812 07:10:53.284039 8210 leader_election.cc:302] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22f9182024ca449898128e8c395f8609; no voters:
I20260812 07:10:53.284614 8210 leader_election.cc:288] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:53.284720 8221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:53.285358 8221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [term 1 LEADER]: Becoming Leader. State: Replica: 22f9182024ca449898128e8c395f8609, State: Running, Role: LEADER
I20260812 07:10:53.285848 8221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f9182024ca449898128e8c395f8609" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45305 } }
I20260812 07:10:53.286427 8210 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:53.288467 8223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22f9182024ca449898128e8c395f8609. Latest consensus state: current_term: 1 leader_uuid: "22f9182024ca449898128e8c395f8609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f9182024ca449898128e8c395f8609" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45305 } } }
I20260812 07:10:53.288844 8223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:53.289420 8222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22f9182024ca449898128e8c395f8609" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22f9182024ca449898128e8c395f8609" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45305 } } }
I20260812 07:10:53.289816 8222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:53.291107 8242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:53.295275 8242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:53.302484 8242 catalog_manager.cc:1347] Generated new cluster ID: 57653155ad83451a81eabd58437ae260
I20260812 07:10:53.302623 8242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:53.336776 8242 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:53.337159 8242 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:53.343792 8242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22f9182024ca449898128e8c395f8609: Generated new TSK 0
I20260812 07:10:53.344595 8242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:10:53.361454 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45305
--builtin_ntp_servers=127.21.181.84:34041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260812 07:10:53.682822 8261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:53.683064 8261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:53.683198 8261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:53.690327 8261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:53.690415 8261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:53.690546 8261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:53.699144 8261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:53.700248 8261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:53.701869 8261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:53.714054 8270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:53.714148 8272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:53.714092 8274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:53.714897 8261 server_base.cc:1034] running on GCE node
I20260812 07:10:53.715480 8261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:53.716363 8261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:53.717561 8261 hybrid_clock.cc:648] HybridClock initialized: now 1786518653717513 us; error 43 us; skew 500 ppm
I20260812 07:10:53.717845 8261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:53.719898 8261 webserver.cc:460] Webserver started at http://127.21.181.65:38787/ using document root <none> and password file <none>
I20260812 07:10:53.720571 8261 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:53.720671 8261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:53.720968 8261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:53.723588 8261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "699344c22c704316b412c1521ebd282b"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-lgr5"
server_key: "4a39ab1a68c3ac8892bfb5527366feec"
server_key_iv: "8e47d461692570b0b851ee5289654db0"
server_key_version: "encryptionkey@0"
I20260812 07:10:53.724388 8261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "699344c22c704316b412c1521ebd282b"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-lgr5"
server_key: "4a39ab1a68c3ac8892bfb5527366feec"
server_key_iv: "8e47d461692570b0b851ee5289654db0"
server_key_version: "encryptionkey@0"
I20260812 07:10:53.729176 8261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:10:53.732432 8283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:53.733641 8261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260812 07:10:53.733788 8261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "699344c22c704316b412c1521ebd282b"
format_stamp: "Formatted at 2026-08-12 07:10:53 on dist-test-slave-lgr5"
server_key: "4a39ab1a68c3ac8892bfb5527366feec"
server_key_iv: "8e47d461692570b0b851ee5289654db0"
server_key_version: "encryptionkey@0"
I20260812 07:10:53.733976 8261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:53.739339 8261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:53.740027 8261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:53.740306 8261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:53.741189 8261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:53.742465 8261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:53.742558 8261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:53.742659 8261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:53.742727 8261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:53.770882 8261 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:40915
I20260812 07:10:53.770936 8494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:40915 every 8 connection(s)
I20260812 07:10:53.772399 8261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:53.777662 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 8261
I20260812 07:10:53.777832 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:53.778613 22229 external_mini_cluster.cc:1428] Setting key 6013813042e986a2b8959f78594cd4c6
I20260812 07:10:53.780187 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:53.774702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:46273 (local address 127.21.181.126:45305)
0812 07:10:53.775148 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:53.775152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:53.776163 (+ 1011us) server_negotiation.cc:407] Connection header received
0812 07:10:53.777809 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:53.777815 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:53.777876 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:53.777993 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:53.779387 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:53.779393 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:53.779398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:53.779418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:53.779462 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:53.779474 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:53.779583 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:53.779781 (+ 198us) server_negotiation.cc:299] Negotiation successful
0812 07:10:53.779858 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":241,"threads_started":1}
I20260812 07:10:53.781081 8498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:53.775134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45305 (local address 127.21.181.65:46273)
0812 07:10:53.775989 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:53.776031 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:53.777547 (+ 1516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:53.778245 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:53.778257 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:53.778382 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:53.779047 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:53.779050 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:53.779098 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:53.779204 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:53.779575 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:53.779585 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:53.779593 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:10:53.779836 (+ 243us) client_negotiation.cc:240] Negotiation successful
0812 07:10:53.780058 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":264,"threads_started":1}
I20260812 07:10:53.781596 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45305
--builtin_ntp_servers=127.21.181.84:34041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 07:10:53.782469 8495 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45305
I20260812 07:10:53.782722 8495 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:53.783242 8495 heartbeater.cc:511] Master 127.21.181.126:45305 requested a full tablet report, sending...
I20260812 07:10:53.785118 8155 ts_manager.cc:194] Registered new tserver with Master: 699344c22c704316b412c1521ebd282b (127.21.181.65:40915)
W20260812 07:10:54.099543 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:54.099861 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:54.100003 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:54.106920 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:54.107017 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:54.107161 8502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:54.115706 8502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:54.116842 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:54.118449 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:54.130330 8510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:54.130373 8511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:54.130388 8516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:54.131455 8502 server_base.cc:1034] running on GCE node
I20260812 07:10:54.131984 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:54.132872 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:54.134074 8502 hybrid_clock.cc:648] HybridClock initialized: now 1786518654134027 us; error 38 us; skew 500 ppm
I20260812 07:10:54.134361 8502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:54.136471 8502 webserver.cc:460] Webserver started at http://127.21.181.66:33253/ using document root <none> and password file <none>
I20260812 07:10:54.137113 8502 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:54.137209 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:54.137531 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:54.140202 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "3ec22ccc2f0444129bd7afdcffcbf47f"
format_stamp: "Formatted at 2026-08-12 07:10:54 on dist-test-slave-lgr5"
server_key: "50a9ff218560676ed16c16bea11e249c"
server_key_iv: "89549758c9fe220744cd394391712476"
server_key_version: "encryptionkey@0"
I20260812 07:10:54.140940 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "3ec22ccc2f0444129bd7afdcffcbf47f"
format_stamp: "Formatted at 2026-08-12 07:10:54 on dist-test-slave-lgr5"
server_key: "50a9ff218560676ed16c16bea11e249c"
server_key_iv: "89549758c9fe220744cd394391712476"
server_key_version: "encryptionkey@0"
I20260812 07:10:54.145553 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:10:54.148635 8533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:54.149796 8502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:54.149951 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "3ec22ccc2f0444129bd7afdcffcbf47f"
format_stamp: "Formatted at 2026-08-12 07:10:54 on dist-test-slave-lgr5"
server_key: "50a9ff218560676ed16c16bea11e249c"
server_key_iv: "89549758c9fe220744cd394391712476"
server_key_version: "encryptionkey@0"
I20260812 07:10:54.150138 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:54.155386 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:54.156136 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:54.156386 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:54.157217 8502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:54.158501 8502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:54.158596 8502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:54.158704 8502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:54.158777 8502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:54.188189 8502 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:44477
I20260812 07:10:54.188262 8740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:44477 every 8 connection(s)
I20260812 07:10:54.189419 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:54.196144 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 8502
I20260812 07:10:54.196307 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:54.196885 22229 external_mini_cluster.cc:1428] Setting key 7a83d50baf4a4d44fb463c948b340eb6
I20260812 07:10:54.196993 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.191794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:50723 (local address 127.21.181.126:45305)
0812 07:10:54.191997 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.192004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.193331 (+ 1327us) server_negotiation.cc:407] Connection header received
0812 07:10:54.194889 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.194893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.194944 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.195032 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.196242 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:54.196248 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:54.196253 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:54.196271 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:54.196310 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:54.196325 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:54.196441 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:54.196634 (+ 193us) server_negotiation.cc:299] Negotiation successful
0812 07:10:54.196698 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:54.197738 8745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.192333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45305 (local address 127.21.181.66:50723)
0812 07:10:54.193183 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:54.193218 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:54.194659 (+ 1441us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:54.195229 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:54.195236 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:54.195317 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:54.195932 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:54.195935 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:54.195971 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:54.196050 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:54.196442 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:54.196453 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:54.196461 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:10:54.196666 (+ 205us) client_negotiation.cc:240] Negotiation successful
0812 07:10:54.196838 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":250,"threads_started":1}
I20260812 07:10:54.199024 8741 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45305
I20260812 07:10:54.199287 8741 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:54.199703 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45305
--builtin_ntp_servers=127.21.181.84:34041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260812 07:10:54.199833 8741 heartbeater.cc:511] Master 127.21.181.126:45305 requested a full tablet report, sending...
I20260812 07:10:54.201323 8152 ts_manager.cc:194] Registered new tserver with Master: 3ec22ccc2f0444129bd7afdcffcbf47f (127.21.181.66:44477)
W20260812 07:10:54.521566 8749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:54.521824 8749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:54.521955 8749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:54.529294 8749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:54.529389 8749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:54.529548 8749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:54.538403 8749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:54.539549 8749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:54.541285 8749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:54.555153 8764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:54.555251 8770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:54.555397 8762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:54.555953 8749 server_base.cc:1034] running on GCE node
I20260812 07:10:54.556548 8749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:54.557387 8749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:54.558583 8749 hybrid_clock.cc:648] HybridClock initialized: now 1786518654558542 us; error 34 us; skew 500 ppm
I20260812 07:10:54.558874 8749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:10:54.560916 8749 webserver.cc:460] Webserver started at http://127.21.181.67:36121/ using document root <none> and password file <none>
I20260812 07:10:54.561539 8749 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:54.561636 8749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:54.561939 8749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:54.564683 8749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "701b8e664dbb450ea391c3db378d005a"
format_stamp: "Formatted at 2026-08-12 07:10:54 on dist-test-slave-lgr5"
server_key: "acd44ea415117460d57748d91fd24510"
server_key_iv: "aa80fc509f050d9820c306c508c6d5bd"
server_key_version: "encryptionkey@0"
I20260812 07:10:54.565424 8749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "701b8e664dbb450ea391c3db378d005a"
format_stamp: "Formatted at 2026-08-12 07:10:54 on dist-test-slave-lgr5"
server_key: "acd44ea415117460d57748d91fd24510"
server_key_iv: "aa80fc509f050d9820c306c508c6d5bd"
server_key_version: "encryptionkey@0"
I20260812 07:10:54.570077 8749 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:10:54.573204 8780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:54.574400 8749 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:10:54.574546 8749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "701b8e664dbb450ea391c3db378d005a"
format_stamp: "Formatted at 2026-08-12 07:10:54 on dist-test-slave-lgr5"
server_key: "acd44ea415117460d57748d91fd24510"
server_key_iv: "aa80fc509f050d9820c306c508c6d5bd"
server_key_version: "encryptionkey@0"
I20260812 07:10:54.574728 8749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:54.579916 8749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:54.580668 8749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:54.580896 8749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:54.581714 8749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:54.582976 8749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:54.583076 8749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:54.583184 8749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:54.583254 8749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:54.612803 8749 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:45783
I20260812 07:10:54.612859 8987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:45783 every 8 connection(s)
I20260812 07:10:54.614010 8749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:54.615187 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 8749
I20260812 07:10:54.615355 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:54.615911 22229 external_mini_cluster.cc:1428] Setting key 86fe648e3f3b5e4aff5d62f335f86f3a
I20260812 07:10:54.621891 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.616651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:60697 (local address 127.21.181.126:45305)
0812 07:10:54.616829 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.616834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.618319 (+ 1485us) server_negotiation.cc:407] Connection header received
0812 07:10:54.620099 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.620106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.620166 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.620247 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.621373 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:54.621377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:54.621380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:54.621392 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:54.621417 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:54.621428 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:54.621520 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:54.621651 (+ 131us) server_negotiation.cc:299] Negotiation successful
0812 07:10:54.621697 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:10:54.622946 8992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.617184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45305 (local address 127.21.181.67:60697)
0812 07:10:54.618194 (+ 1010us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:54.618237 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:54.619825 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:54.620440 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:54.620448 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:54.620533 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:54.621133 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:10:54.621136 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:54.621170 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:10:54.621246 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:54.621502 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:54.621507 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:54.621512 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:10:54.621707 (+ 195us) client_negotiation.cc:240] Negotiation successful
0812 07:10:54.621881 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":292,"threads_started":1}
I20260812 07:10:54.624179 8989 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45305
I20260812 07:10:54.624410 8989 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:54.624908 8989 heartbeater.cc:511] Master 127.21.181.126:45305 requested a full tablet report, sending...
I20260812 07:10:54.626154 8152 ts_manager.cc:194] Registered new tserver with Master: 701b8e664dbb450ea391c3db378d005a (127.21.181.67:45783)
I20260812 07:10:54.630903 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260812 07:10:54.636771 8497 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:10:54.635293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.21.181.126:45305)
0812 07:10:54.635442 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.635446 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.635907 (+ 461us) server_negotiation.cc:407] Connection header received
0812 07:10:54.636016 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.636019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.636065 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.636185 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.636577 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49734: BlockingRecv error: recv got EOF from 127.0.0.1:49734 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:10:54.637465 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 8261
I20260812 07:10:54.640566 9001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.638344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.21.181.65:40915)
0812 07:10:54.638840 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.638846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.638869 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:10:54.638922 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.638928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.639087 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.639196 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.639958 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:54.639964 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:54.639974 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:54.639991 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:54.640028 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:54.640041 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:54.640207 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:54.640262 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 07:10:54.640336 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":239,"threads_started":1}
I20260812 07:10:54.641618 8488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:54.720973 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 8261
I20260812 07:10:54.739939 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 8502
I20260812 07:10:54.743311 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.740882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34910 (local address 127.21.181.66:44477)
0812 07:10:54.741419 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.741426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.741452 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:10:54.741524 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.741530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.741736 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.741854 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.742696 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:54.742702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:54.742713 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:54.742730 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:54.742768 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:54.742781 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:54.742880 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:54.742935 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 07:10:54.743019 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":261,"threads_started":1}
I20260812 07:10:54.744585 8726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:54.823531 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 8502
I20260812 07:10:54.850618 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 8749
I20260812 07:10:54.854444 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:54.851639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.21.181.67:45783)
0812 07:10:54.852369 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:54.852376 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:54.852407 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:10:54.852469 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:54.852475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:54.852686 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:54.852808 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:54.853820 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:54.853827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:54.853838 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:10:54.853850 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:54.853893 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:10:54.853906 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:54.854043 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:54.854091 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 07:10:54.854177 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":368,"threads_started":1}
I20260812 07:10:54.855665 8982 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:54.935240 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 8749
I20260812 07:10:54.960440 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 8101
I20260812 07:10:54.961614 8202 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:55.059819 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 8101
2026-08-12T07:10:55Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2278 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:55 dist-test-slave-lgr5 krb5kdc[9017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:55 dist-test-slave-lgr5 krb5kdc[9017](info): set up 2 sockets
Aug 12 07:10:55 dist-test-slave-lgr5 krb5kdc[9017](info): commencing operation
krb5kdc: starting...
W20260812 07:10:56.332832 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:10:56 dist-test-slave-lgr5 krb5kdc[9017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:10:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:10:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:10:56.571101 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44455
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:44455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:56.885079 9038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:56.885342 9038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:56.885479 9038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:56.892373 9038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:10:56.892467 9038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:56.892551 9038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:56.892628 9038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:10:56.892688 9038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:10:56.901954 9038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:44455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:56.903102 9038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:56.904390 9038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:56.915297 9043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.915346 9051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:56.915370 9049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:56.916144 9038 server_base.cc:1034] running on GCE node
I20260812 07:10:56.916906 9038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:56.918202 9038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:56.919406 9038 hybrid_clock.cc:648] HybridClock initialized: now 1786518656919347 us; error 45 us; skew 500 ppm
I20260812 07:10:56.919715 9038 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:56 dist-test-slave-lgr5 krb5kdc[9017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:56.926740 9038 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:10:56.928092 9038 webserver.cc:460] Webserver started at http://127.21.181.126:44507/ using document root <none> and password file <none>
I20260812 07:10:56.928727 9038 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:56.928843 9038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:56.929136 9038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:56.931895 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "096f3c7d8e2f4cfb9857f7198340b551"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-lgr5"
server_key: "618f4bc128dd7f254af152fedc1f0b00"
server_key_iv: "7bae46dee60423bb77583315a3935594"
server_key_version: "encryptionkey@0"
I20260812 07:10:56.932655 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "096f3c7d8e2f4cfb9857f7198340b551"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-lgr5"
server_key: "618f4bc128dd7f254af152fedc1f0b00"
server_key_iv: "7bae46dee60423bb77583315a3935594"
server_key_version: "encryptionkey@0"
I20260812 07:10:56.937404 9038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:10:56.940599 9065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:56.941797 9038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260812 07:10:56.941958 9038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "096f3c7d8e2f4cfb9857f7198340b551"
format_stamp: "Formatted at 2026-08-12 07:10:56 on dist-test-slave-lgr5"
server_key: "618f4bc128dd7f254af152fedc1f0b00"
server_key_iv: "7bae46dee60423bb77583315a3935594"
server_key_version: "encryptionkey@0"
I20260812 07:10:56.942137 9038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:56.963456 9038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:56.967357 9038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:56.967605 9038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:56.983453 9038 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:44455
I20260812 07:10:56.983490 9163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:44455 every 8 connection(s)
I20260812 07:10:56.984696 9038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:10:56.988137 9165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:56.989645 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 9038
I20260812 07:10:56.989822 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:10:56.990448 22229 external_mini_cluster.cc:1428] Setting key 4ba561eb02f7550f60db78d4f635212a
I20260812 07:10:56.999567 9165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551: Bootstrap starting.
I20260812 07:10:57.003154 9165 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:57.004413 9165 log.cc:824] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:10:57 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:57.007259 9165 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551: No bootstrap required, opened a new log
I20260812 07:10:57.012704 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:56.992354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53040 (local address 127.21.181.126:44455)
0812 07:10:56.993122 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:56.993133 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:56.993175 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:10:56.994774 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:56.994795 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:56.995168 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:56.995470 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:56.997351 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:56.998622 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.000430 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.001077 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.001198 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.005869 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:57.005876 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:57.005890 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:57.005923 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:57.008870 (+ 2947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.009631 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.009637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.009642 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.009804 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.010197 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.010200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.010202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.010696 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:57.010938 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:57.011338 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 07:10:57.011682 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":359,"threads_started":1}
I20260812 07:10:57.016040 9165 raft_consensus.cc:348] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096f3c7d8e2f4cfb9857f7198340b551" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44455 } }
I20260812 07:10:57.016418 9165 raft_consensus.cc:374] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:57.016518 9165 raft_consensus.cc:729] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096f3c7d8e2f4cfb9857f7198340b551, State: Initialized, Role: FOLLOWER
I20260812 07:10:57.017069 9165 consensus_queue.cc:260] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096f3c7d8e2f4cfb9857f7198340b551" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44455 } }
I20260812 07:10:57.017298 9165 raft_consensus.cc:388] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:10:57.017442 9165 raft_consensus.cc:482] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:10:57.017635 9165 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:57.019622 9165 raft_consensus.cc:504] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096f3c7d8e2f4cfb9857f7198340b551" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44455 } }
I20260812 07:10:57.020097 9165 leader_election.cc:302] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 096f3c7d8e2f4cfb9857f7198340b551; no voters:
I20260812 07:10:57.020705 9165 leader_election.cc:288] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:10:57.020802 9173 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:57.021476 9173 raft_consensus.cc:686] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [term 1 LEADER]: Becoming Leader. State: Replica: 096f3c7d8e2f4cfb9857f7198340b551, State: Running, Role: LEADER
I20260812 07:10:57.022018 9173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096f3c7d8e2f4cfb9857f7198340b551" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44455 } }
I20260812 07:10:57.022791 9165 sys_catalog.cc:564] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:10:57.024736 9174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "096f3c7d8e2f4cfb9857f7198340b551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096f3c7d8e2f4cfb9857f7198340b551" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44455 } } }
I20260812 07:10:57.025120 9174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:57.025666 9177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 096f3c7d8e2f4cfb9857f7198340b551. Latest consensus state: current_term: 1 leader_uuid: "096f3c7d8e2f4cfb9857f7198340b551" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "096f3c7d8e2f4cfb9857f7198340b551" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44455 } } }
I20260812 07:10:57.025993 9177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551 [sys.catalog]: This master's current role is: LEADER
I20260812 07:10:57.032747 9188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:10:57.036841 9188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:10:57.044831 9188 catalog_manager.cc:1347] Generated new cluster ID: 20dd8775990e48d89c3f19a6e0e52a76
I20260812 07:10:57.044938 9188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:10:57.065968 9188 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:10:57.067768 9188 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:10:57.082993 9188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551: Generated new TSK 0
I20260812 07:10:57.083977 9188 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:10:57.147956 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44455
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:57.481308 9210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:57.481601 9210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:57.481761 9210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:57.489029 9210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:57.489151 9210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:57.489333 9210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:10:57.498939 9210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:57.500218 9210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:57.501514 9210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:57.514611 9218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:57.514662 9219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:57.514694 9229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:57.515581 9210 server_base.cc:1034] running on GCE node
I20260812 07:10:57.516266 9210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:57.517164 9210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:57.518388 9210 hybrid_clock.cc:648] HybridClock initialized: now 1786518657518342 us; error 31 us; skew 500 ppm
I20260812 07:10:57.518694 9210 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:57 dist-test-slave-lgr5 krb5kdc[9017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:57.521857 9210 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:10:57.523236 9210 webserver.cc:460] Webserver started at http://127.21.181.65:37947/ using document root <none> and password file <none>
I20260812 07:10:57.523869 9210 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:57.523989 9210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:57.524335 9210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:57.527122 9210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "63663da6f0b348368d736db28718243f"
format_stamp: "Formatted at 2026-08-12 07:10:57 on dist-test-slave-lgr5"
server_key: "8e586a267098357c3913891c7e906de9"
server_key_iv: "542f3921db4846f84985622899c49824"
server_key_version: "encryptionkey@0"
I20260812 07:10:57.527839 9210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "63663da6f0b348368d736db28718243f"
format_stamp: "Formatted at 2026-08-12 07:10:57 on dist-test-slave-lgr5"
server_key: "8e586a267098357c3913891c7e906de9"
server_key_iv: "542f3921db4846f84985622899c49824"
server_key_version: "encryptionkey@0"
I20260812 07:10:57.532553 9210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:10:57.535806 9240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.537081 9210 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:10:57.537225 9210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "63663da6f0b348368d736db28718243f"
format_stamp: "Formatted at 2026-08-12 07:10:57 on dist-test-slave-lgr5"
server_key: "8e586a267098357c3913891c7e906de9"
server_key_iv: "542f3921db4846f84985622899c49824"
server_key_version: "encryptionkey@0"
I20260812 07:10:57.537392 9210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:57.550729 9210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:57.554148 9210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:57.554399 9210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:57.555265 9210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:57.556540 9210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:57.556614 9210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.556725 9210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:57.556782 9210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:57.587138 9210 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:42137
I20260812 07:10:57.587180 9435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:42137 every 8 connection(s)
I20260812 07:10:57.588400 9210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:10:57.595376 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 9210
I20260812 07:10:57.595600 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:10:57.596351 22229 external_mini_cluster.cc:1428] Setting key a472400c5ab21f561339a33654ba47c3
Aug 12 07:10:57 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518657, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:57.609143 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.590770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:53051 (local address 127.21.181.126:44455)
0812 07:10:57.591198 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:57.591202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:57.592161 (+ 959us) server_negotiation.cc:407] Connection header received
0812 07:10:57.593933 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:57.593939 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:57.594018 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:57.594117 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:57.595928 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.596732 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.598626 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.599031 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.599139 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.603141 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:57.603148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:57.603152 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:57.603188 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:57.606032 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.606810 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.606816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.606819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.606955 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:57.607399 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:57.607403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:57.607406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:57.607725 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:57.607868 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:57.608604 (+ 736us) server_negotiation.cc:299] Negotiation successful
0812 07:10:57.608805 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":208,"threads_started":1}
I20260812 07:10:57.610025 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:57.591293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44455 (local address 127.21.181.65:53051)
0812 07:10:57.591994 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:57.592030 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:57.593663 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:57.594367 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:57.594374 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:57.594861 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:57.595738 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.595768 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.596894 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.596899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.598427 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:57.598444 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:57.599266 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:57.599272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:57.599528 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:57.600238 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:57.600266 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:57.602907 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:57.606169 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.606178 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.606191 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.606678 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.607064 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:57.607068 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:57.607071 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:57.607282 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:57.607838 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:57.607844 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:57.608215 (+ 371us) client_negotiation.cc:769] Sending connection context
0812 07:10:57.608478 (+ 263us) client_negotiation.cc:240] Negotiation successful
0812 07:10:57.608928 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":274,"threads_started":1}
I20260812 07:10:57.611477 9437 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44455
I20260812 07:10:57.611756 9437 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:57.612515 9437 heartbeater.cc:511] Master 127.21.181.126:44455 requested a full tablet report, sending...
I20260812 07:10:57.614748 9104 ts_manager.cc:194] Registered new tserver with Master: 63663da6f0b348368d736db28718243f (127.21.181.65:42137)
I20260812 07:10:57.616673 9104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:53051
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:10:57.656253 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44455
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:57.968019 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:57.968290 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:57.968427 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:57.977141 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:57.977236 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:57.977401 9454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:10:57.986618 9454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:57.987748 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:57.988974 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:58.001439 9461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:58.001386 9465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:58.001468 9462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:58.002185 9454 server_base.cc:1034] running on GCE node
I20260812 07:10:58.002763 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:58.003639 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:58.004843 9454 hybrid_clock.cc:648] HybridClock initialized: now 1786518658004789 us; error 43 us; skew 500 ppm
I20260812 07:10:58.005133 9454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:58 dist-test-slave-lgr5 krb5kdc[9017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:58.008306 9454 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:10:58.009704 9454 webserver.cc:460] Webserver started at http://127.21.181.66:36143/ using document root <none> and password file <none>
I20260812 07:10:58.010331 9454 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:58.010456 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:58.010730 9454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:58.013510 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "d4bcf764ab094976b233adf6b0a71ab1"
format_stamp: "Formatted at 2026-08-12 07:10:58 on dist-test-slave-lgr5"
server_key: "df693667403616ee44b13d507461a221"
server_key_iv: "50721b909f2e0e67ff37168a7082ba7a"
server_key_version: "encryptionkey@0"
I20260812 07:10:58.014225 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "d4bcf764ab094976b233adf6b0a71ab1"
format_stamp: "Formatted at 2026-08-12 07:10:58 on dist-test-slave-lgr5"
server_key: "df693667403616ee44b13d507461a221"
server_key_iv: "50721b909f2e0e67ff37168a7082ba7a"
server_key_version: "encryptionkey@0"
I20260812 07:10:58.019083 9454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:10:58.022342 9478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:58.023643 9454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:10:58.023782 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "d4bcf764ab094976b233adf6b0a71ab1"
format_stamp: "Formatted at 2026-08-12 07:10:58 on dist-test-slave-lgr5"
server_key: "df693667403616ee44b13d507461a221"
server_key_iv: "50721b909f2e0e67ff37168a7082ba7a"
server_key_version: "encryptionkey@0"
I20260812 07:10:58.023944 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:58.039573 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:58.043335 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:58.043602 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:58.044488 9454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:58.045779 9454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:58.045877 9454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:58.045996 9454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:58.046041 9454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:58.076236 9454 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:36181
I20260812 07:10:58.076279 9675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:36181 every 8 connection(s)
I20260812 07:10:58.077400 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:10:58.086174 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 9454
I20260812 07:10:58.086333 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:10:58.086760 22229 external_mini_cluster.cc:1428] Setting key f5431c4d6a1c3cc46e9b177a5e4b880b
Aug 12 07:10:58 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:58.095955 9440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.079778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:45123 (local address 127.21.181.126:44455)
0812 07:10:58.079946 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.079950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.081103 (+ 1153us) server_negotiation.cc:407] Connection header received
0812 07:10:58.082846 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.082852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.082931 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.083027 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:58.084518 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.085217 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.086777 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.087226 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.087343 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.090612 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:58.090617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:58.090620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:58.090653 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:58.093093 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.093781 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.093785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.093788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.093911 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.094330 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.094333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.094335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.094595 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:58.094742 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.095420 (+ 678us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.095640 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:10:58.096808 9681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.080241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44455 (local address 127.21.181.66:45123)
0812 07:10:58.080939 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.080984 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.082579 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.083270 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.083278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.083762 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:58.084341 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.084361 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.085376 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.085380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.086631 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.086644 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.087449 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.087458 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.087678 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.088336 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:58.088357 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:58.090392 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:58.093230 (+ 2838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:58.093238 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:58.093252 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:58.093652 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:58.094022 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:58.094027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:58.094031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:58.094206 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:58.094729 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:58.094736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:58.095028 (+ 292us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.095276 (+ 248us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.095740 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":264,"threads_started":1}
I20260812 07:10:58.098161 9677 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44455
I20260812 07:10:58.098423 9677 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:58.099099 9677 heartbeater.cc:511] Master 127.21.181.126:44455 requested a full tablet report, sending...
I20260812 07:10:58.100539 9104 ts_manager.cc:194] Registered new tserver with Master: d4bcf764ab094976b233adf6b0a71ab1 (127.21.181.66:36181)
I20260812 07:10:58.101562 9104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:45123
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:10:58.145192 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44455
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:33075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:10:58.464213 9685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:10:58.464468 9685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:10:58.464558 9685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:10:58.471539 9685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:10:58.471639 9685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:10:58.471755 9685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:10:58.481251 9685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:10:58.482425 9685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:10:58.483675 9685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:10:58.496323 9691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:58.496373 9692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:10:58.496376 9696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:10:58.497233 9685 server_base.cc:1034] running on GCE node
I20260812 07:10:58.497828 9685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:10:58.498668 9685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:10:58.499882 9685 hybrid_clock.cc:648] HybridClock initialized: now 1786518658499826 us; error 48 us; skew 500 ppm
I20260812 07:10:58.500223 9685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:10:58 dist-test-slave-lgr5 krb5kdc[9017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:10:58.503198 9685 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:10:58.504670 9685 webserver.cc:460] Webserver started at http://127.21.181.67:42355/ using document root <none> and password file <none>
I20260812 07:10:58.505324 9685 fs_manager.cc:359] Metadata directory not provided
I20260812 07:10:58.505439 9685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:10:58.505725 9685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:10:58.508741 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "fe0af524d70f4cecb2ca4f8212d9bc61"
format_stamp: "Formatted at 2026-08-12 07:10:58 on dist-test-slave-lgr5"
server_key: "c21539f4f570e1aa8c18501bc7248033"
server_key_iv: "b22fc4864b38643c4fd9ba7f507744a1"
server_key_version: "encryptionkey@0"
I20260812 07:10:58.509485 9685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "fe0af524d70f4cecb2ca4f8212d9bc61"
format_stamp: "Formatted at 2026-08-12 07:10:58 on dist-test-slave-lgr5"
server_key: "c21539f4f570e1aa8c18501bc7248033"
server_key_iv: "b22fc4864b38643c4fd9ba7f507744a1"
server_key_version: "encryptionkey@0"
I20260812 07:10:58.514369 9685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:10:58.517717 9708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:58.519044 9685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:10:58.519191 9685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "fe0af524d70f4cecb2ca4f8212d9bc61"
format_stamp: "Formatted at 2026-08-12 07:10:58 on dist-test-slave-lgr5"
server_key: "c21539f4f570e1aa8c18501bc7248033"
server_key_iv: "b22fc4864b38643c4fd9ba7f507744a1"
server_key_version: "encryptionkey@0"
I20260812 07:10:58.519356 9685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:10:58.547946 9685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:10:58.555193 9685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:10:58.555486 9685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:10:58.556524 9685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:10:58.558228 9685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:10:58.558321 9685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:58.558444 9685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:10:58.558516 9685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:10:58.602299 9685 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:35229
I20260812 07:10:58.602391 9916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:35229 every 8 connection(s)
I20260812 07:10:58.603559 9685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:10:58.612689 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 9685
I20260812 07:10:58.612836 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:10:58.613270 22229 external_mini_cluster.cc:1428] Setting key e83f13dedf5acb80a6327a31ed0eaa19
Aug 12 07:10:58 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518658, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:10:58.619486 9437 heartbeater.cc:503] Master 127.21.181.126:44455 was elected leader, sending a full tablet report...
I20260812 07:10:58.623016 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.606042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57465 (local address 127.21.181.126:44455)
0812 07:10:58.606525 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.606529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.607280 (+ 751us) server_negotiation.cc:407] Connection header received
0812 07:10:58.609327 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.609333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.609426 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.609539 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:58.611066 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.611871 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.613628 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.614077 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.614208 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.617704 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:58.617709 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:58.617712 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:58.617744 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:58.620210 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.620947 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.620953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.620956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.621114 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.621556 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.621561 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.621563 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.621835 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:58.621952 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.622532 (+ 580us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.622698 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":246,"threads_started":1}
I20260812 07:10:58.623994 9925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.606404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44455 (local address 127.21.181.67:57465)
0812 07:10:58.607117 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.607164 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.609032 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.609795 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.609807 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.610311 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:10:58.610864 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.610882 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.612041 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.612045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.613467 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.613482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.614327 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.614336 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.614576 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.615258 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:10:58.615284 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:10:58.617490 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:10:58.620339 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:58.620345 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:58.620355 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:58.620781 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:58.621230 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:10:58.621234 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:10:58.621236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:10:58.621412 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:10:58.621937 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:10:58.621942 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:10:58.622208 (+ 266us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.622412 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.622867 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":266,"threads_started":1}
I20260812 07:10:58.625247 9921 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44455
I20260812 07:10:58.625548 9921 heartbeater.cc:464] Registering TS with master...
I20260812 07:10:58.626219 9921 heartbeater.cc:511] Master 127.21.181.126:44455 requested a full tablet report, sending...
I20260812 07:10:58.627676 9104 ts_manager.cc:194] Registered new tserver with Master: fe0af524d70f4cecb2ca4f8212d9bc61 (127.21.181.67:35229)
I20260812 07:10:58.628430 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:10:58.628687 9104 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:57465
I20260812 07:10:58.644729 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.632993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53048 (local address 127.21.181.126:44455)
0812 07:10:58.633142 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.633146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.633628 (+ 482us) server_negotiation.cc:407] Connection header received
0812 07:10:58.633774 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.633778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.633836 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.633912 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:58.635233 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.635923 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.637176 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.637580 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.637687 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.639822 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:58.639827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:58.639830 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:58.639860 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:58.642247 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.642862 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.642866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.642868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.642983 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.643352 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.643355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.643357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.643655 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:58.643880 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.644237 (+ 357us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.644440 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:10:58.648509 9104 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:10:58.655210 9104 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:10:58.677743 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.666664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35229 (local address 127.0.0.1:40844)
0812 07:10:58.667590 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.667645 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.667816 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.670168 (+ 2352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.670173 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.670199 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.670629 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.670642 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.672807 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.672813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.674901 (+ 2088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.674914 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.676708 (+ 1794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.676714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.676921 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.676980 (+ 59us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.677095 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.677273 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":3,"thread_start_us":1484,"threads_started":1}
I20260812 07:10:58.677981 9942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.666964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36181 (local address 127.0.0.1:47838)
0812 07:10:58.668398 (+ 1434us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.668415 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.668520 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.669599 (+ 1079us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.669607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.669646 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.670080 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.670094 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.672208 (+ 2114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.672214 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.674704 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.674720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.676681 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.676692 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.677250 (+ 558us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.677280 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.677357 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.677478 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1324,"mutex_wait_us":405,"thread_start_us":812,"threads_started":1}
I20260812 07:10:58.678964 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.667455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47838 (local address 127.21.181.66:36181)
0812 07:10:58.668965 (+ 1510us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.668986 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.669017 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:10:58.669108 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.669115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.669363 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.670271 (+ 908us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.670501 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.672019 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.674939 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.676445 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.676925 (+ 480us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.677136 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.678511 (+ 1375us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.678681 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1348,"thread_start_us":1498,"threads_started":1}
I20260812 07:10:58.679005 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.667055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40844 (local address 127.21.181.67:35229)
0812 07:10:58.669612 (+ 2557us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.669620 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.669651 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:10:58.669741 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.669748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.669959 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.670838 (+ 879us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.671112 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.672654 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.675093 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.676445 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.678239 (+ 1794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.678467 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.678526 (+ 59us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.678677 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2374,"thread_start_us":2211,"threads_started":1}
I20260812 07:10:58.683001 9790 tablet_service.cc:1467] Processing CreateTablet for tablet e03d3089b93c46d4bc19d647578f4ccd (DEFAULT_TABLE table=test-table [id=d5af5408a8eb4a2e8eebb018edcfd206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:58.683704 9556 tablet_service.cc:1467] Processing CreateTablet for tablet e03d3089b93c46d4bc19d647578f4ccd (DEFAULT_TABLE table=test-table [id=d5af5408a8eb4a2e8eebb018edcfd206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:58.684269 9790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e03d3089b93c46d4bc19d647578f4ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:58.685089 9556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e03d3089b93c46d4bc19d647578f4ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:58.686931 9943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.668682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:42137 (local address 127.0.0.1:56706)
0812 07:10:58.677975 (+ 9293us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.677992 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.678087 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.680298 (+ 2211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.680303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.680329 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.680504 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.680515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.682608 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.682613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.684417 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.684432 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.686212 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.686218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.686390 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.686422 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.686535 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.686670 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9203,"thread_start_us":11987,"threads_started":1}
I20260812 07:10:58.687083 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.669296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56706 (local address 127.21.181.65:42137)
0812 07:10:58.679703 (+ 10407us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.679712 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.679738 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:10:58.679826 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.679833 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.680060 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.680686 (+ 626us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.680967 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.682440 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.684611 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.686020 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.686164 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.686382 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.686594 (+ 212us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.686759 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10237,"thread_start_us":10083,"threads_started":1}
I20260812 07:10:58.690696 9329 tablet_service.cc:1467] Processing CreateTablet for tablet e03d3089b93c46d4bc19d647578f4ccd (DEFAULT_TABLE table=test-table [id=d5af5408a8eb4a2e8eebb018edcfd206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:10:58.692036 9329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e03d3089b93c46d4bc19d647578f4ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:10:58.696002 9951 tablet_bootstrap.cc:492] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Bootstrap starting.
I20260812 07:10:58.699375 9951 tablet_bootstrap.cc:654] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:58.699836 9952 tablet_bootstrap.cc:492] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Bootstrap starting.
I20260812 07:10:58.700824 9951 log.cc:824] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:58.703090 9952 tablet_bootstrap.cc:654] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:58.704010 9951 tablet_bootstrap.cc:492] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: No bootstrap required, opened a new log
I20260812 07:10:58.704309 9951 ts_tablet_manager.cc:1397] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.005s
I20260812 07:10:58.704349 9952 log.cc:824] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:58.705075 9957 tablet_bootstrap.cc:492] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Bootstrap starting.
I20260812 07:10:58.708303 9957 tablet_bootstrap.cc:654] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Neither blocks nor log segments found. Creating new log.
I20260812 07:10:58.708519 9952 tablet_bootstrap.cc:492] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: No bootstrap required, opened a new log
I20260812 07:10:58.708808 9952 ts_tablet_manager.cc:1397] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260812 07:10:58.709244 9957 log.cc:824] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:10:58.711583 9957 tablet_bootstrap.cc:492] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: No bootstrap required, opened a new log
I20260812 07:10:58.711798 9957 ts_tablet_manager.cc:1397] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:10:58.714123 9951 raft_consensus.cc:348] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.714541 9951 raft_consensus.cc:374] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:58.714637 9951 raft_consensus.cc:729] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0af524d70f4cecb2ca4f8212d9bc61, State: Initialized, Role: FOLLOWER
I20260812 07:10:58.715265 9951 consensus_queue.cc:260] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.716362 9921 heartbeater.cc:503] Master 127.21.181.126:44455 was elected leader, sending a full tablet report...
I20260812 07:10:58.718601 9952 raft_consensus.cc:348] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.719035 9952 raft_consensus.cc:374] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:58.719141 9952 raft_consensus.cc:729] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4bcf764ab094976b233adf6b0a71ab1, State: Initialized, Role: FOLLOWER
I20260812 07:10:58.719467 9957 raft_consensus.cc:348] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.719794 9957 raft_consensus.cc:374] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:10:58.719894 9957 raft_consensus.cc:729] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63663da6f0b348368d736db28718243f, State: Initialized, Role: FOLLOWER
I20260812 07:10:58.719723 9952 consensus_queue.cc:260] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.720554 9957 consensus_queue.cc:260] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.721338 9951 ts_tablet_manager.cc:1428] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Time spent starting tablet: real 0.017s user 0.006s sys 0.007s
I20260812 07:10:58.722136 9957 ts_tablet_manager.cc:1428] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260812 07:10:58.723140 9677 heartbeater.cc:503] Master 127.21.181.126:44455 was elected leader, sending a full tablet report...
I20260812 07:10:58.724226 9952 ts_tablet_manager.cc:1428] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260812 07:10:58.736651 9962 raft_consensus.cc:482] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:10:58.736867 9962 raft_consensus.cc:504] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.738243 9962 leader_election.cc:288] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe0af524d70f4cecb2ca4f8212d9bc61 (127.21.181.67:35229), 63663da6f0b348368d736db28718243f (127.21.181.65:42137)
I20260812 07:10:58.744792 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.738662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:47585 (local address 127.21.181.67:35229)
0812 07:10:58.738849 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.738855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.739547 (+ 692us) server_negotiation.cc:407] Connection header received
0812 07:10:58.739721 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.739725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.739782 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.739854 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.740464 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.741235 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.743091 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.743828 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.743972 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.744061 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.744431 (+ 370us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.744555 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:10:58.744881 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.738671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:42591 (local address 127.21.181.65:42137)
0812 07:10:58.738851 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.738856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.739606 (+ 750us) server_negotiation.cc:407] Connection header received
0812 07:10:58.739759 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.739763 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.739815 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.739886 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:10:58.740587 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.741396 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.743191 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.743948 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.744115 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.744190 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.744530 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.744653 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:10:58.745080 9964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.738627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35229 (local address 127.21.181.66:47585)
0812 07:10:58.739433 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.739447 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.739590 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.739889 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.739894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.739919 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.740324 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.740335 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.741365 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.741370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.742934 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.742947 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.744042 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.744048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.744222 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.744297 (+ 75us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.744383 (+ 86us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.744515 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"mutex_wait_us":44,"thread_start_us":439,"threads_started":1}
I20260812 07:10:58.745260 9967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.738678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:42137 (local address 127.21.181.66:42591)
0812 07:10:58.739495 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0812 07:10:58.739514 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:10:58.739633 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:10:58.739918 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:10:58.739923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:10:58.739946 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:10:58.740407 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.740417 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.741538 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.741545 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.743033 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:10:58.743047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.744190 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.744197 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:10:58.744372 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.744394 (+ 22us) client_negotiation.cc:769] Sending connection context
0812 07:10:58.744471 (+ 77us) client_negotiation.cc:240] Negotiation successful
0812 07:10:58.744587 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":15,"thread_start_us":428,"threads_started":1}
I20260812 07:10:58.746176 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03d3089b93c46d4bc19d647578f4ccd" candidate_uuid: "d4bcf764ab094976b233adf6b0a71ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" is_pre_election: true
I20260812 07:10:58.746265 9368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03d3089b93c46d4bc19d647578f4ccd" candidate_uuid: "d4bcf764ab094976b233adf6b0a71ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63663da6f0b348368d736db28718243f" is_pre_election: true
I20260812 07:10:58.746712 9826 raft_consensus.cc:2454] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bcf764ab094976b233adf6b0a71ab1 in term 0.
I20260812 07:10:58.746763 9368 raft_consensus.cc:2454] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4bcf764ab094976b233adf6b0a71ab1 in term 0.
I20260812 07:10:58.747375 9482 leader_election.cc:302] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4bcf764ab094976b233adf6b0a71ab1, fe0af524d70f4cecb2ca4f8212d9bc61; no voters:
I20260812 07:10:58.747912 9962 raft_consensus.cc:2781] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:10:58.748112 9962 raft_consensus.cc:482] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:10:58.748245 9962 raft_consensus.cc:3037] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:58.750564 9962 raft_consensus.cc:504] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.751183 9962 leader_election.cc:288] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [CANDIDATE]: Term 1 election: Requested vote from peers fe0af524d70f4cecb2ca4f8212d9bc61 (127.21.181.67:35229), 63663da6f0b348368d736db28718243f (127.21.181.65:42137)
I20260812 07:10:58.751629 9826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03d3089b93c46d4bc19d647578f4ccd" candidate_uuid: "d4bcf764ab094976b233adf6b0a71ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61"
I20260812 07:10:58.751822 9826 raft_consensus.cc:3037] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:58.751874 9368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e03d3089b93c46d4bc19d647578f4ccd" candidate_uuid: "d4bcf764ab094976b233adf6b0a71ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63663da6f0b348368d736db28718243f"
I20260812 07:10:58.752058 9368 raft_consensus.cc:3037] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:10:58.754180 9826 raft_consensus.cc:2454] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bcf764ab094976b233adf6b0a71ab1 in term 1.
I20260812 07:10:58.754233 9368 raft_consensus.cc:2454] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4bcf764ab094976b233adf6b0a71ab1 in term 1.
I20260812 07:10:58.754596 9483 leader_election.cc:302] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63663da6f0b348368d736db28718243f, d4bcf764ab094976b233adf6b0a71ab1; no voters:
I20260812 07:10:58.754944 9962 raft_consensus.cc:2781] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:10:58.755407 9962 raft_consensus.cc:686] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 1 LEADER]: Becoming Leader. State: Replica: d4bcf764ab094976b233adf6b0a71ab1, State: Running, Role: LEADER
I20260812 07:10:58.755915 9962 consensus_queue.cc:237] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } }
I20260812 07:10:58.760244 9103 catalog_manager.cc:5495] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4bcf764ab094976b233adf6b0a71ab1 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "d4bcf764ab094976b233adf6b0a71ab1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4bcf764ab094976b233adf6b0a71ab1" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 } health_report { overall_health: UNKNOWN } } }
I20260812 07:10:58.777546 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.772507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47842 (local address 127.21.181.66:36181)
0812 07:10:58.772699 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.772704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.772719 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:10:58.772913 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.772916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.772983 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.773097 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:10:58.773426 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.774088 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.775472 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.775858 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.775942 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.776453 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:10:58.776895 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:10:58.776997 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.777170 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.777265 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:10:58.785090 9826 raft_consensus.cc:1261] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 1 FOLLOWER]: Refusing update from remote peer d4bcf764ab094976b233adf6b0a71ab1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:58.785104 9368 raft_consensus.cc:1261] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 1 FOLLOWER]: Refusing update from remote peer d4bcf764ab094976b233adf6b0a71ab1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:10:58.785939 9972 consensus_queue.cc:1035] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe0af524d70f4cecb2ca4f8212d9bc61" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35229 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:58.786273 9962 consensus_queue.cc:1035] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63663da6f0b348368d736db28718243f" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 42137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:10:58.797531 9976 mvcc.cc:205] Tried to move back new op lower bound from 7317580426374737920 to 7317580426268045312. Current Snapshot: MvccSnapshot[applied={T|T < 7317580426374737920}]
I20260812 07:10:58.801098 9977 mvcc.cc:205] Tried to move back new op lower bound from 7317580426374737920 to 7317580426268045312. Current Snapshot: MvccSnapshot[applied={T|T < 7317580426374737920}]
I20260812 07:10:58.809720 9103 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:58.809995 9103 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:10:58.813771 9103 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 096f3c7d8e2f4cfb9857f7198340b551: Sending DeleteTablet for 3 replicas of tablet e03d3089b93c46d4bc19d647578f4ccd
I20260812 07:10:58.814874 9790 tablet_service.cc:1514] Processing DeleteTablet for tablet e03d3089b93c46d4bc19d647578f4ccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:58 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:40844
I20260812 07:10:58.815325 9556 tablet_service.cc:1514] Processing DeleteTablet for tablet e03d3089b93c46d4bc19d647578f4ccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:58 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:47838
I20260812 07:10:58.815434 9329 tablet_service.cc:1514] Processing DeleteTablet for tablet e03d3089b93c46d4bc19d647578f4ccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:10:58 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56706
I20260812 07:10:58.817189 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 9210
I20260812 07:10:58.827802 9990 tablet_replica.cc:330] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: stopping tablet replica
I20260812 07:10:58.828346 9990 raft_consensus.cc:2229] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:10:58.829094 9990 raft_consensus.cc:2258] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:58.830124 9991 tablet_replica.cc:330] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: stopping tablet replica
I20260812 07:10:58.830506 9991 raft_consensus.cc:2229] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:58.830989 9991 raft_consensus.cc:2258] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:58.831722 9986 tablet_replica.cc:330] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: stopping tablet replica
I20260812 07:10:58.832113 9986 raft_consensus.cc:2229] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:10:58.832557 9986 raft_consensus.cc:2258] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:10:58.833328 9991 ts_tablet_manager.cc:1905] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:58.833606 9990 ts_tablet_manager.cc:1905] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:58.835117 9986 ts_tablet_manager.cc:1905] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:10:58.840478 9991 ts_tablet_manager.cc:1918] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:58.840674 9991 log.cc:1196] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/e03d3089b93c46d4bc19d647578f4ccd
I20260812 07:10:58.841329 9991 ts_tablet_manager.cc:1939] T e03d3089b93c46d4bc19d647578f4ccd P 63663da6f0b348368d736db28718243f: Deleting consensus metadata
I20260812 07:10:58.841568 9990 ts_tablet_manager.cc:1918] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:58.841835 9990 log.cc:1196] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/e03d3089b93c46d4bc19d647578f4ccd
I20260812 07:10:58.842540 9990 ts_tablet_manager.cc:1939] T e03d3089b93c46d4bc19d647578f4ccd P d4bcf764ab094976b233adf6b0a71ab1: Deleting consensus metadata
I20260812 07:10:58.843327 9067 catalog_manager.cc:4841] TS 63663da6f0b348368d736db28718243f (127.21.181.65:42137): tablet e03d3089b93c46d4bc19d647578f4ccd (table test-table [id=d5af5408a8eb4a2e8eebb018edcfd206]) successfully deleted
I20260812 07:10:58.844399 9067 catalog_manager.cc:4841] TS d4bcf764ab094976b233adf6b0a71ab1 (127.21.181.66:36181): tablet e03d3089b93c46d4bc19d647578f4ccd (table test-table [id=d5af5408a8eb4a2e8eebb018edcfd206]) successfully deleted
I20260812 07:10:58.845839 9986 ts_tablet_manager.cc:1918] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:10:58.845986 9986 log.cc:1196] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/e03d3089b93c46d4bc19d647578f4ccd
I20260812 07:10:58.846494 9986 ts_tablet_manager.cc:1939] T e03d3089b93c46d4bc19d647578f4ccd P fe0af524d70f4cecb2ca4f8212d9bc61: Deleting consensus metadata
Aug 12 07:10:58 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:10:58.847970 9066 catalog_manager.cc:4841] TS fe0af524d70f4cecb2ca4f8212d9bc61 (127.21.181.67:35229): tablet e03d3089b93c46d4bc19d647578f4ccd (table test-table [id=d5af5408a8eb4a2e8eebb018edcfd206]) successfully deleted
I20260812 07:10:58.855502 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:58.819062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56708 (local address 127.21.181.65:42137)
0812 07:10:58.819264 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:58.819272 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:58.829644 (+ 10372us) server_negotiation.cc:407] Connection header received
0812 07:10:58.829738 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:58.829745 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:58.829876 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:58.830003 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:58.840026 (+ 10023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.841241 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.843163 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:58.843793 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:58.843916 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:58.849030 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:58.849044 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:58.849061 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:58.849101 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:58.852670 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.853344 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.853353 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.853356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.853529 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:58.853925 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:58.853930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:58.853933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:58.854414 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:58.854722 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:58.854895 (+ 173us) server_negotiation.cc:299] Negotiation successful
0812 07:10:58.855145 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:10:58.856218 9431 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:58.977247 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 9210
I20260812 07:10:59.003217 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 9454
Aug 12 07:10:59 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:10:59.018616 9948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:59.003955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.21.181.66:36181)
0812 07:10:59.004234 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:59.004239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:59.004262 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:10:59.004483 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:59.004487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:59.004581 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:59.004666 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:59.005896 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:59.006695 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:59.008019 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:59.008635 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:59.008750 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:59.012930 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:59.012944 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:59.012959 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:59.012996 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:59.015874 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:59.016667 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:59.016674 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:59.016676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:59.016822 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:59.017221 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:59.017224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:59.017226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:59.017583 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:59.017837 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:59.018133 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 07:10:59.018297 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260812 07:10:59.019343 9670 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:59.129204 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 9454
I20260812 07:10:59.156463 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 9685
Aug 12 07:10:59 dist-test-slave-lgr5 krb5kdc[9017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:10:59.170080 9947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:10:59.157141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.21.181.67:35229)
0812 07:10:59.157305 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:10:59.157309 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:10:59.157335 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:10:59.157492 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:10:59.157496 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:10:59.157570 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:10:59.157655 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:10:59.158923 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:59.159642 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:59.160866 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:10:59.161261 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:10:59.161357 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:10:59.165359 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:10:59.165373 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:10:59.165389 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:10:59.165421 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:10:59.167852 (+ 2431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:59.168472 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:59.168479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:59.168481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:59.168629 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:10:59.168921 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:10:59.168923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:10:59.168926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:10:59.169254 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:10:59.169474 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:10:59.169680 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 07:10:59.169843 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:10:59.170718 9906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:59.257964 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 9685
I20260812 07:10:59.282193 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 9038
I20260812 07:10:59.283314 9155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:10:59.393342 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 9038
2026-08-12T07:10:59Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4331 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:10:59 dist-test-slave-lgr5 krb5kdc[10012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:10:59 dist-test-slave-lgr5 krb5kdc[10012](info): set up 2 sockets
Aug 12 07:10:59 dist-test-slave-lgr5 krb5kdc[10012](info): commencing operation
krb5kdc: starting...
W20260812 07:11:00.643810 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:11:00 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518660, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:11:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:11:00Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/rangeradmin_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/rangeradmin_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/rangerlookup_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/rangerlookup_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
I20260812 07:11:00.978889 22229 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCYQ_m9/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/postgres -l logfile start
2026-08-12 07:11:03.181 UTC [10054] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 07:11:03.181 UTC [10054] LOG: listening on IPv4 address "127.21.181.84", port 53301
2026-08-12 07:11:03.184 UTC [10054] LOG: listening on Unix socket "/tmp/.s.PGSQL.53301"
2026-08-12 07:11:03.205 UTC [10057] LOG: database system was shut down at 2026-08-12 07:11:02 UTC
2026-08-12 07:11:03.208 UTC [10054] LOG: database system is ready to accept connections
I20260812 07:11:04.334726 22229 mini_postgres.cc:96] Postgres bound to 53301
2026-08-12 07:11:04.357 UTC [10079] FATAL: database "slave" does not exist
127.21.181.84:53301 - accepting connections
I20260812 07:11:04.358637 22229 mini_ranger.cc:161] Starting Ranger...
I20260812 07:11:04.389292 22229 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 07:11:04.787305 22229 mini_ranger.cc:87] Created ranger Postgres database
I20260812 07:11:04.787451 22229 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-admin
2026-08-12 07:11:05,125 [I] DB FLAVOR :POSTGRES
2026-08-12 07:11:05,125 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:11:05,125 [I] Checking connection..
2026-08-12 07:11:05,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 07:11:05,391 [I] Checking connection passed.
2026-08-12 07:11:05,391 [I] --------- Verifying version history table ---------
2026-08-12 07:11:05,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:11:05,642 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:11:05,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:11:05,892 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:11:05,893 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 07:11:05,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 07:11:06.141 UTC [10191] WARNING: there is no transaction in progress
2026-08-12 07:11:06,150 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 07:11:06,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:11:06,414 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 07:11:06,415 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:11:06,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 07:11:06,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 07:11:06,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lgr5.c.gcp-upstream.internal','N') ;"
2026-08-12 07:11:07,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:11:07,378 [I] Table x_portal_user does not exist in database ranger
2026-08-12 07:11:07,378 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:11:07,635 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 07:11:07,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:11:07,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:11:08,123 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 07:11:08,123 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 07:11:08.973 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:08.988 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.001 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.015 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.031 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.048 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.098 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.103 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.115 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.124 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.133 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.145 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.152 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.157 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.163 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09.168 UTC [10363] WARNING: there is no transaction in progress
2026-08-12 07:11:09,684 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 07:11:09,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 07:11:09,932 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 07:11:09,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:11:10,170 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 07:11:10,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:11:10,394 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 07:11:10,394 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:11:10,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:11:10,894 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lgr5.c.gcp-upstream.internal';"
2026-08-12 07:11:11,127 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 07:11:11,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 07:11:11,373 [I] DB_PATCHES have already been applied
I20260812 07:11:11.379316 22229 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCYQ_m9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 07:11:11.379424 22229 mini_ranger.cc:193] Using host: 127.21.181.84
I20260812 07:11:11.399803 22229 mini_ranger.cc:227] Ranger admin URL: http://127.21.181.84:38389
Aug 12, 2026 7:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:11:11 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:11:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:11:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 07:11:12.673079 22229 mini_ranger.cc:160] Time spent starting Ranger: real 8.314s user 0.004s sys 0.053s
Aug 12, 2026 7:11:31 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:11:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 12, 2026 7:11:32 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:11:32 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:11:32 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 07:11:33.502591 22229 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/rangerkms_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/rangerkms_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.181.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.181.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 07:11:33 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518693, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 07:11:33.681902 22229 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 07:11:33.701284 22229 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 07:11:34.363780 22229 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 07:11:34.363899 22229 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms
I20260812 07:11:34.364058 22229 mini_ranger_kms.cc:227] Using postgres at 127.21.181.84:53301
2026-08-12 07:11:34,795 [I] DB FLAVOR :POSTGRES
2026-08-12 07:11:34,795 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:11:34,795 [I] Checking connection
2026-08-12 07:11:34,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 07:11:35,028 [I] connection success
2026-08-12 07:11:35,029 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 07:11:35,029 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 07:11:35,029 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 07:11:35,262 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 07:11:35,262 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:11:35,262 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 07:11:35,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:53301/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 07:11:35,517 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 07:11:35.939347 22229 mini_ranger_kms.cc:310] Created kms service
I20260812 07:11:36.121524 22229 mini_ranger_kms.cc:326] Created kudu user
I20260812 07:11:36.163137 22229 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 07:11:36 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518693, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:11:36.567019 22229 mini_ranger_kms.cc:384] Added ranger policy
I20260812 07:11:36.567355 22229 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 07:11:36.567426 22229 mini_ranger_kms.cc:240] Using host: 127.21.181.84
I20260812 07:11:36.592451 22229 mini_ranger_kms.cc:276] Ranger KMS PID: 10664
I20260812 07:11:36.592597 22229 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.181.84:43933
Aug 12, 2026 7:11:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:11:37 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:11:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:11:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:11:37 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:11:37 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 07:11:38 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518698, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.181.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 12 07:11:39 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518698, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.181.84@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:11:39.229529 22229 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.548s user 0.125s sys 0.340s
I20260812 07:11:39.229803 22229 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 07:11:39.230026 22229 mini_ranger_kms.cc:401] 127.21.181.84:43933/kms/v1/keys
Aug 12 07:11:39 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518698, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.181.84@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
Aug 12 07:11:41 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:11:41.789640 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39537
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:39537
--ranger_config_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:43933/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:11:42.118148 10740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:42.118399 10740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:42.118527 10740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:42.125730 10740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:11:42.125837 10740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:42.125912 10740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:42.125986 10740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:11:42.126049 10740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:11:42.135895 10740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.181.84:43933/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:39537
--ranger_config_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:11:42.137055 10740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:42.138299 10740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:42.149784 10751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:42.149762 10752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:42.149717 10754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:42.150827 10740 server_base.cc:1034] running on GCE node
I20260812 07:11:42.151516 10740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:42.152828 10740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:42.154028 10740 hybrid_clock.cc:648] HybridClock initialized: now 1786518702153983 us; error 31 us; skew 500 ppm
I20260812 07:11:42.154349 10740 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:11:42 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518702, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:42.157320 10740 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:11:42.158630 10740 webserver.cc:460] Webserver started at http://127.21.181.126:43049/ using document root <none> and password file <none>
I20260812 07:11:42.159214 10740 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:42.159323 10740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:42.159606 10740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:11:42 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518702, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:11:42.208351 10740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "39310523fc574d98a5eaadea9e5cdd3f"
format_stamp: "Formatted at 2026-08-12 07:11:42 on dist-test-slave-lgr5"
server_key: "c96fb8b0b5e187997222054aba4029c2"
server_key_iv: "74b79d33c28a10ba0d213b3ccdf24f9b"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:42.209466 10740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "39310523fc574d98a5eaadea9e5cdd3f"
format_stamp: "Formatted at 2026-08-12 07:11:42 on dist-test-slave-lgr5"
server_key: "c96fb8b0b5e187997222054aba4029c2"
server_key_iv: "74b79d33c28a10ba0d213b3ccdf24f9b"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:42.216436 10740 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260812 07:11:42.233101 10771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:42.234670 10740 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:11:42.234838 10740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "39310523fc574d98a5eaadea9e5cdd3f"
format_stamp: "Formatted at 2026-08-12 07:11:42 on dist-test-slave-lgr5"
server_key: "c96fb8b0b5e187997222054aba4029c2"
server_key_iv: "74b79d33c28a10ba0d213b3ccdf24f9b"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:42.235055 10740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:42.261351 10740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:42.265142 10740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:42.265434 10740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:42.284030 10740 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:39537
I20260812 07:11:42.284053 10865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:39537 every 8 connection(s)
I20260812 07:11:42.285418 10740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:11:42.289156 10870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:42.294409 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 10740
I20260812 07:11:42.294634 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:11:42.295473 22229 external_mini_cluster.cc:1428] Setting key e345929a9fcbadb358082f60906a03e8
I20260812 07:11:42.300330 10870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f: Bootstrap starting.
I20260812 07:11:42.304352 10870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:42.305751 10870 log.cc:824] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:42.314512 10870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f: No bootstrap required, opened a new log
I20260812 07:11:42.323545 10870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39310523fc574d98a5eaadea9e5cdd3f" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39537 } }
I20260812 07:11:42.323980 10870 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:42.324108 10870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39310523fc574d98a5eaadea9e5cdd3f, State: Initialized, Role: FOLLOWER
Aug 12 07:11:42 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:11:42.325273 10870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39310523fc574d98a5eaadea9e5cdd3f" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39537 } }
I20260812 07:11:42.325559 10870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:11:42.325712 10870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:11:42.325906 10870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:42.328387 10870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39310523fc574d98a5eaadea9e5cdd3f" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39537 } }
I20260812 07:11:42.328861 10870 leader_election.cc:302] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39310523fc574d98a5eaadea9e5cdd3f; no voters:
I20260812 07:11:42.329522 10870 leader_election.cc:288] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:11:42.329607 10879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:42.330312 10879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [term 1 LEADER]: Becoming Leader. State: Replica: 39310523fc574d98a5eaadea9e5cdd3f, State: Running, Role: LEADER
I20260812 07:11:42.330930 10879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39310523fc574d98a5eaadea9e5cdd3f" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39537 } }
I20260812 07:11:42.333068 10870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:11:42.333757 10876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:42.297577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59684 (local address 127.21.181.126:39537)
0812 07:11:42.312343 (+ 14766us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:42.312358 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:42.312409 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:11:42.314836 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:42.314867 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:42.315319 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:42.315705 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:42.317278 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:42.318779 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:42.320564 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:42.321282 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:42.321419 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:42.325826 (+ 4407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:42.325836 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:42.325854 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:42.325901 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:42.328827 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:42.329676 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:42.329688 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:42.329697 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:42.329896 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:42.330345 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:42.330350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:42.330352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:42.331172 (+ 820us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:42.331474 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:42.331913 (+ 439us) server_negotiation.cc:299] Negotiation successful
0812 07:11:42.332341 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14514,"thread_start_us":14775,"threads_started":1}
I20260812 07:11:42.334066 10880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39310523fc574d98a5eaadea9e5cdd3f. Latest consensus state: current_term: 1 leader_uuid: "39310523fc574d98a5eaadea9e5cdd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39310523fc574d98a5eaadea9e5cdd3f" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39537 } } }
I20260812 07:11:42.334443 10880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:42.334954 10881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39310523fc574d98a5eaadea9e5cdd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39310523fc574d98a5eaadea9e5cdd3f" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39537 } } }
I20260812 07:11:42.335229 10881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f [sys.catalog]: This master's current role is: LEADER
I20260812 07:11:42.338876 10884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:11:42.344259 10884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:11:42.344954 10870 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:11:42.362612 10884 catalog_manager.cc:1347] Generated new cluster ID: 7e752426c4c340968f10961023ff7853
I20260812 07:11:42.362802 10884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:11:42.372920 10884 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:11:42.374787 10884 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:11:42.385098 10884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f: Generated new TSK 0
I20260812 07:11:42.385826 10884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:11:43.179663 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:43933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39537
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:11:43 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 07:11:43.600333 10944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:43.600713 10944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:43.600884 10944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:43.611931 10944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:43.612097 10944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:43.612296 10944 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:11:43.627498 10944 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.181.84:43933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:11:43.629401 10944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:43.631237 10944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:43.650529 10963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:43.651979 10964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:43.660364 10944 server_base.cc:1034] running on GCE node
W20260812 07:11:43.661089 10969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:43.661998 10944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:43.663239 10944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:43.664531 10944 hybrid_clock.cc:648] HybridClock initialized: now 1786518703664454 us; error 62 us; skew 500 ppm
I20260812 07:11:43.664927 10944 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:11:43 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:43.669281 10944 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:11:43.671241 10944 webserver.cc:460] Webserver started at http://127.21.181.65:44731/ using document root <none> and password file <none>
I20260812 07:11:43.672171 10944 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:43.672315 10944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:43.672696 10944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:11:43 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:11:43.695860 10944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "e34fdede2ab54eb48593dcf1b13deb65"
format_stamp: "Formatted at 2026-08-12 07:11:43 on dist-test-slave-lgr5"
server_key: "4ee1a1a09da94bd2502938c43eb2d486"
server_key_iv: "ca791914c6c9e897471cfedd0fada91e"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:43.697023 10944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "e34fdede2ab54eb48593dcf1b13deb65"
format_stamp: "Formatted at 2026-08-12 07:11:43 on dist-test-slave-lgr5"
server_key: "4ee1a1a09da94bd2502938c43eb2d486"
server_key_iv: "ca791914c6c9e897471cfedd0fada91e"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:43.713799 10944 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.010s sys 0.005s
I20260812 07:11:43.739076 10981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:43.741128 10944 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.010s sys 0.000s
I20260812 07:11:43.741446 10944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "e34fdede2ab54eb48593dcf1b13deb65"
format_stamp: "Formatted at 2026-08-12 07:11:43 on dist-test-slave-lgr5"
server_key: "4ee1a1a09da94bd2502938c43eb2d486"
server_key_iv: "ca791914c6c9e897471cfedd0fada91e"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:43.741784 10944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:43.801013 10944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:43.806007 10944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:43.806414 10944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:43.812516 10944 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:43.814082 10944 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:43.814167 10944 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:43.814288 10944 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:43.814342 10944 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:43.880760 10944 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:45729
I20260812 07:11:43.880806 11172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:45729 every 8 connection(s)
I20260812 07:11:43.882478 10944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:11:43.890211 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 10944
I20260812 07:11:43.890429 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:11:43.891230 22229 external_mini_cluster.cc:1428] Setting key 64cb8b8ab78361f87a0312ee1498feac
Aug 12 07:11:43 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:11:43.954587 11179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:43.886637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39537 (local address 127.21.181.65:41241)
0812 07:11:43.932431 (+ 45794us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:43.932500 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:43.934977 (+ 2477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:43.935865 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:43.935876 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:43.936497 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:43.937262 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:43.937292 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:43.938831 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:43.938836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:43.941001 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:43.941028 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:43.942063 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:43.942072 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:43.942368 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:43.943148 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:43.943178 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:43.946099 (+ 2921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:43.949967 (+ 3868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:43.949975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:43.949989 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:43.950516 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:43.950969 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:43.950973 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:43.950977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:43.951214 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:43.951935 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:43.951943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:43.952385 (+ 442us) client_negotiation.cc:769] Sending connection context
0812 07:11:43.952694 (+ 309us) client_negotiation.cc:240] Negotiation successful
0812 07:11:43.953191 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45390,"thread_start_us":45152,"threads_started":1}
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
I20260812 07:11:43.956432 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:43.885949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:41241 (local address 127.21.181.126:39537)
0812 07:11:43.927309 (+ 41360us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:43.927317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:43.933064 (+ 5747us) server_negotiation.cc:407] Connection header received
0812 07:11:43.935375 (+ 2311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:43.935382 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:43.935485 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:43.935598 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:43.937462 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:43.938672 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:43.941155 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:43.941819 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:43.941940 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:43.946353 (+ 4413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:43.946360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:43.946365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:43.946413 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:43.949820 (+ 3407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:43.950678 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:43.950684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:43.950688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:43.950835 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:43.951362 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:43.951367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:43.951371 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:43.951717 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:43.951859 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:43.955780 (+ 3921us) server_negotiation.cc:299] Negotiation successful
0812 07:11:43.956016 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41187,"thread_start_us":41368,"threads_started":1}
I20260812 07:11:43.958135 11174 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39537
I20260812 07:11:43.958477 11174 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:43.959458 11174 heartbeater.cc:511] Master 127.21.181.126:39537 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
I20260812 07:11:43.961982 10794 ts_manager.cc:194] Registered new tserver with Master: e34fdede2ab54eb48593dcf1b13deb65 (127.21.181.65:45729)
I20260812 07:11:43.963799 10794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:41241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
W20260812 07:11:44.000909 22229 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.21.181.66: real 0.107s user 0.000s sys 0.024s
I20260812 07:11:44.002018 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:43933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39537
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:11:44 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
Aug 12 07:11:44 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518703, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
W20260812 07:11:44.425374 11182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:44.425649 11182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:44.425825 11182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:44.435329 11182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:44.435549 11182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:44.435807 11182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:11:44.450942 11182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.181.84:43933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:11:44.452831 11182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:44.454632 11182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:44.483765 11192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:44.486236 11194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:44.504329 11182 server_base.cc:1034] running on GCE node
W20260812 07:11:44.505244 11201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:44.506112 11182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:44.512370 11182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:44.513625 11182 hybrid_clock.cc:648] HybridClock initialized: now 1786518704513554 us; error 50 us; skew 500 ppm
I20260812 07:11:44.514048 11182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:11:44 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518704, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:44.519042 11182 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:11:44.548521 11182 webserver.cc:460] Webserver started at http://127.21.181.66:36345/ using document root <none> and password file <none>
I20260812 07:11:44.549978 11182 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:44.550197 11182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:44.550637 11182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:11:44 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518704, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:11:44.569517 11182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "785d7f0c6e6647bb819ac5923380ff59"
format_stamp: "Formatted at 2026-08-12 07:11:44 on dist-test-slave-lgr5"
server_key: "ef38e16c1da10f660199efbb0f2430d8"
server_key_iv: "3612a3b83c1a10b8f6e02108d70008b0"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:44.570670 11182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "785d7f0c6e6647bb819ac5923380ff59"
format_stamp: "Formatted at 2026-08-12 07:11:44 on dist-test-slave-lgr5"
server_key: "ef38e16c1da10f660199efbb0f2430d8"
server_key_iv: "3612a3b83c1a10b8f6e02108d70008b0"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:44.580013 11182 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260812 07:11:44.615789 11212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:44.617861 11182 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.005s sys 0.007s
I20260812 07:11:44.618088 11182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "785d7f0c6e6647bb819ac5923380ff59"
format_stamp: "Formatted at 2026-08-12 07:11:44 on dist-test-slave-lgr5"
server_key: "ef38e16c1da10f660199efbb0f2430d8"
server_key_iv: "3612a3b83c1a10b8f6e02108d70008b0"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:44.618372 11182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:44.639099 11182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:44.643100 11182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:44.643380 11182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:44.644434 11182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:44.646131 11182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:44.646210 11182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:44.646335 11182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:44.646394 11182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:44.695515 11182 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:39851
I20260812 07:11:44.695539 11409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:39851 every 8 connection(s)
I20260812 07:11:44.697197 11182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:11:44.699931 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 11182
I20260812 07:11:44.700120 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:11:44.700699 22229 external_mini_cluster.cc:1428] Setting key c512cb46378b254c2bb3c591250e1af2
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Aug 12 07:11:44 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518704, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:11:44.748791 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:44.700150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:36235 (local address 127.21.181.126:39537)
0812 07:11:44.716986 (+ 16836us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:44.716992 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:44.727956 (+ 10964us) server_negotiation.cc:407] Connection header received
0812 07:11:44.728035 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:44.728040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:44.728160 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:44.728267 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:44.730729 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:44.731964 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:44.734457 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:44.735121 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:44.735252 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:44.741652 (+ 6400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:44.741659 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:44.741664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:44.741703 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:44.745056 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:44.745919 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:44.745925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:44.745928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:44.746079 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:44.746594 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:44.746599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:44.746602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:44.746984 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:44.747127 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:44.748170 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0812 07:11:44.748426 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16679,"thread_start_us":16833,"threads_started":1}
I20260812 07:11:44.750983 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:44.700850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39537 (local address 127.21.181.66:36235)
0812 07:11:44.724764 (+ 23914us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:44.724839 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:44.727559 (+ 2720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:44.728968 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:44.728981 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:44.729632 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:44.730526 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:44.730555 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:44.732170 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:44.732176 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:44.734282 (+ 2106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:44.734300 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:44.735374 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:44.735382 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:44.735682 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:44.736502 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:44.736535 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:44.741411 (+ 4876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:44.745213 (+ 3802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:44.745222 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:44.745238 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:44.745787 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:44.746215 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:44.746220 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:44.746223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:44.746449 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:44.747295 (+ 846us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:44.747302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:44.747721 (+ 419us) client_negotiation.cc:769] Sending connection context
0812 07:11:44.749133 (+ 1412us) client_negotiation.cc:240] Negotiation successful
0812 07:11:44.749654 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23552,"thread_start_us":23288,"threads_started":1}
I20260812 07:11:44.752679 11410 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39537
I20260812 07:11:44.752988 11410 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:44.753834 11410 heartbeater.cc:511] Master 127.21.181.126:39537 requested a full tablet report, sending...
I20260812 07:11:44.755517 10794 ts_manager.cc:194] Registered new tserver with Master: 785d7f0c6e6647bb819ac5923380ff59 (127.21.181.66:39851)
I20260812 07:11:44.756595 10794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:36235
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:11:44.772766 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:43933/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39537
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:40533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:11:44.967239 11174 heartbeater.cc:503] Master 127.21.181.126:39537 was elected leader, sending a full tablet report...
W20260812 07:11:45.111354 11421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:11:45.111622 11421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:11:45.111754 11421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:11:45.118804 11421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:11:45.118912 11421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:11:45.119058 11421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:11:45.128598 11421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:40533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.181.84:43933/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:39537
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:11:45.129750 11421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:11:45.130995 11421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:11:45.143594 11435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:45.143656 11428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:11:45.143677 11429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:11:45.144290 11421 server_base.cc:1034] running on GCE node
I20260812 07:11:45.144867 11421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:11:45.145725 11421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:11:45.146921 11421 hybrid_clock.cc:648] HybridClock initialized: now 1786518705146880 us; error 31 us; skew 500 ppm
I20260812 07:11:45.147256 11421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:11:45 dist-test-slave-lgr5 krb5kdc[10012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518705, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:11:45.150463 11421 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:11:45.151772 11421 webserver.cc:460] Webserver started at http://127.21.181.67:43451/ using document root <none> and password file <none>
I20260812 07:11:45.152438 11421 fs_manager.cc:359] Metadata directory not provided
I20260812 07:11:45.152557 11421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:11:45.152846 11421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:11:45 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518705, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:11:45.167574 11421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "73d14fffc10f493e98d608cbfb696921"
format_stamp: "Formatted at 2026-08-12 07:11:45 on dist-test-slave-lgr5"
server_key: "37d2cb3acc35923644488020583e8407"
server_key_iv: "0844b2d7467ffdac2325b099ca98e7a6"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:45.168385 11421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "73d14fffc10f493e98d608cbfb696921"
format_stamp: "Formatted at 2026-08-12 07:11:45 on dist-test-slave-lgr5"
server_key: "37d2cb3acc35923644488020583e8407"
server_key_iv: "0844b2d7467ffdac2325b099ca98e7a6"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:45.173627 11421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 07:11:45.190320 11446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:45.191768 11421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 07:11:45.191926 11421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "73d14fffc10f493e98d608cbfb696921"
format_stamp: "Formatted at 2026-08-12 07:11:45 on dist-test-slave-lgr5"
server_key: "37d2cb3acc35923644488020583e8407"
server_key_iv: "0844b2d7467ffdac2325b099ca98e7a6"
server_key_version: "kuduclusterkey@0"
I20260812 07:11:45.192157 11421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:11:45.212972 11421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:11:45.216737 11421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:11:45.217041 11421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:11:45.217954 11421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:11:45.219291 11421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:11:45.219367 11421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:45.219476 11421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:11:45.219533 11421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:11:45.250394 11421 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:45913
I20260812 07:11:45.250437 11639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:45913 every 8 connection(s)
I20260812 07:11:45.251626 11421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:11:45.257757 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 11421
I20260812 07:11:45.257894 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:11:45.258340 22229 external_mini_cluster.cc:1428] Setting key 1df8e110e61fb81c6e62aa0a7214ae2d
Aug 12 07:11:45 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518705, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:11:45.270437 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.254167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:47103 (local address 127.21.181.126:39537)
0812 07:11:45.254703 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.254707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.255418 (+ 711us) server_negotiation.cc:407] Connection header received
0812 07:11:45.257273 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.257279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.257361 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.257480 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:45.259013 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.259874 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.261915 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.262359 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.262452 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.265527 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:45.265532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:45.265536 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:45.265567 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:45.267938 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.268595 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.268599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.268601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.268726 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.269091 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.269093 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.269095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.269346 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:45.269453 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.270016 (+ 563us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.270180 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":253,"threads_started":1}
I20260812 07:11:45.271351 11647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.254534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39537 (local address 127.21.181.67:47103)
0812 07:11:45.255257 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:45.255303 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:45.257001 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:45.257711 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:45.257719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:45.258187 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:11:45.258791 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.258818 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.260052 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.260058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.261773 (+ 1715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.261794 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.262541 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.262546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.262733 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.263347 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:11:45.263369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:11:45.265316 (+ 1947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:11:45.268057 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:45.268063 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:45.268074 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:45.268501 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:45.268804 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:11:45.268807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:11:45.268809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:11:45.269009 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:11:45.269433 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:11:45.269438 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:11:45.269725 (+ 287us) client_negotiation.cc:769] Sending connection context
0812 07:11:45.269934 (+ 209us) client_negotiation.cc:240] Negotiation successful
0812 07:11:45.270303 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":277,"threads_started":1}
I20260812 07:11:45.272742 11640 heartbeater.cc:347] Connected to a master server at 127.21.181.126:39537
I20260812 07:11:45.273144 11640 heartbeater.cc:464] Registering TS with master...
I20260812 07:11:45.273931 11640 heartbeater.cc:511] Master 127.21.181.126:39537 requested a full tablet report, sending...
I20260812 07:11:45.275329 10793 ts_manager.cc:194] Registered new tserver with Master: 73d14fffc10f493e98d608cbfb696921 (127.21.181.67:45913)
I20260812 07:11:45.276347 10793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:47103
I20260812 07:11:45.285786 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:11:45.301895 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.290553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59696 (local address 127.21.181.126:39537)
0812 07:11:45.290724 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.290729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.291065 (+ 336us) server_negotiation.cc:407] Connection header received
0812 07:11:45.291191 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.291195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.291272 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.291353 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:45.292836 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.293539 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.294774 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.295191 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.295284 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.297592 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:45.297597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:45.297600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:45.297632 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:45.299872 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.300455 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.300458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.300461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.300575 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.300860 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.300862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.300864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.301119 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:45.301292 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.301531 (+ 239us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.301685 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:11:45.305768 10793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:11:45.313186 10793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:11:45.346429 11668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.325084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:45913 (local address 127.0.0.1:47080)
0812 07:11:45.336586 (+ 11502us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:45.336606 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:45.337180 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:45.338326 (+ 1146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:45.338335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:45.338376 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:45.338840 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.338852 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.341125 (+ 2273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.341131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.343668 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.343683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.345460 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.345466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.345696 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.345763 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 07:11:45.345896 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 07:11:45.346074 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11359,"thread_start_us":11658,"threads_started":1}
I20260812 07:11:45.346447 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.325703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.21.181.67:45913)
0812 07:11:45.326380 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.326389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.336717 (+ 10328us) server_negotiation.cc:407] Connection header received
0812 07:11:45.337399 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.337410 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.337701 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.337923 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:45.339074 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.340887 (+ 1813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.343836 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.345119 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.345191 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.345403 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.345955 (+ 552us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.346121 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488,"thread_start_us":305,"threads_started":1}
I20260812 07:11:45.349712 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.325089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.21.181.65:45729)
0812 07:11:45.341745 (+ 16656us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.341755 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.341783 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:11:45.341884 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.341892 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.342133 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.343549 (+ 1416us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:45.343788 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.345152 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.347407 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.348694 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.348823 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.349053 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.349252 (+ 199us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.349414 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16463,"thread_start_us":17185,"threads_started":1}
I20260812 07:11:45.349717 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.324867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:45729 (local address 127.0.0.1:39414)
0812 07:11:45.325575 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:45.325626 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:45.325804 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:45.342545 (+ 16741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:45.342551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:45.342578 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:45.343071 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.343082 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.345296 (+ 2214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.345302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.347190 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.347209 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.348893 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.348899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.349067 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.349100 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:11:45.349178 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:11:45.349299 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":1,"thread_start_us":312,"threads_started":1}
I20260812 07:11:45.350615 11532 tablet_service.cc:1467] Processing CreateTablet for tablet 8c2f2282f32f48d48559c600a5c8c291 (DEFAULT_TABLE table=test-table [id=6f0e398c54614139abbe0f266d1c0dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:45.352138 11532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c2f2282f32f48d48559c600a5c8c291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:45.352561 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.327362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36924 (local address 127.21.181.66:39851)
0812 07:11:45.343296 (+ 15934us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.343303 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.343720 (+ 417us) server_negotiation.cc:407] Connection header received
0812 07:11:45.343818 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.343824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.344057 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.344275 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:45.344850 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.346276 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.350365 (+ 4089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.351534 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.351663 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.351850 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.352102 (+ 252us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.352265 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15774,"thread_start_us":15929,"threads_started":1}
I20260812 07:11:45.352594 11671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.327155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39851 (local address 127.0.0.1:36924)
0812 07:11:45.343328 (+ 16173us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:45.343344 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:45.343457 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:45.344270 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:45.344274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:45.344300 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:45.344633 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.344645 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.346435 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.346440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.350209 (+ 3769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.350224 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.351735 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.351741 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.351902 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.351933 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:11:45.352037 (+ 104us) client_negotiation.cc:240] Negotiation successful
0812 07:11:45.352178 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16033,"mutex_wait_us":40,"thread_start_us":16388,"threads_started":1}
I20260812 07:11:45.353899 11065 tablet_service.cc:1467] Processing CreateTablet for tablet 8c2f2282f32f48d48559c600a5c8c291 (DEFAULT_TABLE table=test-table [id=6f0e398c54614139abbe0f266d1c0dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:45.355011 11065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c2f2282f32f48d48559c600a5c8c291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:45.356194 11320 tablet_service.cc:1467] Processing CreateTablet for tablet 8c2f2282f32f48d48559c600a5c8c291 (DEFAULT_TABLE table=test-table [id=6f0e398c54614139abbe0f266d1c0dd7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:11:45.357496 11320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c2f2282f32f48d48559c600a5c8c291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:11:45.366801 11678 tablet_bootstrap.cc:492] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Bootstrap starting.
I20260812 07:11:45.368826 11679 tablet_bootstrap.cc:492] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Bootstrap starting.
I20260812 07:11:45.370759 11678 tablet_bootstrap.cc:654] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:45.371749 11681 tablet_bootstrap.cc:492] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Bootstrap starting.
I20260812 07:11:45.372418 11678 log.cc:824] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:45.372975 11679 tablet_bootstrap.cc:654] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:45.374396 11679 log.cc:824] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:45.375550 11681 tablet_bootstrap.cc:654] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Neither blocks nor log segments found. Creating new log.
I20260812 07:11:45.376669 11681 log.cc:824] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Log is configured to *not* fsync() on all Append() calls
I20260812 07:11:45.377027 11678 tablet_bootstrap.cc:492] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: No bootstrap required, opened a new log
I20260812 07:11:45.377254 11678 ts_tablet_manager.cc:1397] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.008s
I20260812 07:11:45.379073 11681 tablet_bootstrap.cc:492] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: No bootstrap required, opened a new log
I20260812 07:11:45.379187 11679 tablet_bootstrap.cc:492] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: No bootstrap required, opened a new log
I20260812 07:11:45.379314 11681 ts_tablet_manager.cc:1397] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 07:11:45.379436 11679 ts_tablet_manager.cc:1397] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260812 07:11:45.386759 11681 raft_consensus.cc:348] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.387151 11681 raft_consensus.cc:374] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:45.387326 11681 raft_consensus.cc:729] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785d7f0c6e6647bb819ac5923380ff59, State: Initialized, Role: FOLLOWER
I20260812 07:11:45.387915 11681 consensus_queue.cc:260] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.387888 11678 raft_consensus.cc:348] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.388317 11678 raft_consensus.cc:374] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:45.388417 11678 raft_consensus.cc:729] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34fdede2ab54eb48593dcf1b13deb65, State: Initialized, Role: FOLLOWER
I20260812 07:11:45.389067 11678 consensus_queue.cc:260] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.389878 11679 raft_consensus.cc:348] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.390303 11679 raft_consensus.cc:374] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:11:45.390417 11679 raft_consensus.cc:729] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d14fffc10f493e98d608cbfb696921, State: Initialized, Role: FOLLOWER
I20260812 07:11:45.391010 11679 consensus_queue.cc:260] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.392405 11678 ts_tablet_manager.cc:1428] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
I20260812 07:11:45.393253 11640 heartbeater.cc:503] Master 127.21.181.126:39537 was elected leader, sending a full tablet report...
I20260812 07:11:45.393848 11679 ts_tablet_manager.cc:1428] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260812 07:11:45.395848 11681 ts_tablet_manager.cc:1428] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Time spent starting tablet: real 0.016s user 0.009s sys 0.009s
I20260812 07:11:45.396023 11410 heartbeater.cc:503] Master 127.21.181.126:39537 was elected leader, sending a full tablet report...
W20260812 07:11:45.434980 11175 tablet.cc:2369] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:11:45.479867 11411 tablet.cc:2369] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:11:45.503173 11643 tablet.cc:2369] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:11:45.595630 11686 raft_consensus.cc:482] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:11:45.595932 11686 raft_consensus.cc:504] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.597946 11686 leader_election.cc:288] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e34fdede2ab54eb48593dcf1b13deb65 (127.21.181.65:45729), 73d14fffc10f493e98d608cbfb696921 (127.21.181.67:45913)
I20260812 07:11:45.607085 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.598307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:55385 (local address 127.21.181.65:45729)
0812 07:11:45.598863 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.598871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.599085 (+ 214us) server_negotiation.cc:407] Connection header received
0812 07:11:45.599311 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.599317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.599397 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.599493 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:45.600541 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.601779 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.604738 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.606136 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.606277 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.606325 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.606708 (+ 383us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.606829 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384}
I20260812 07:11:45.607081 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.598606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:57179 (local address 127.21.181.67:45913)
0812 07:11:45.599032 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.599037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.599283 (+ 246us) server_negotiation.cc:407] Connection header received
0812 07:11:45.599409 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.599413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.599498 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.599582 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:11:45.600678 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.601774 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.604730 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.605959 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.606126 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.606200 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.606713 (+ 513us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.606832 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260812 07:11:45.607400 11694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.598046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:45729 (local address 127.21.181.66:55385)
0812 07:11:45.598962 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:45.598985 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:45.599160 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:45.599518 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:45.599522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:45.599544 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:45.600391 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.600412 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.601897 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.601902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.604594 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.604610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.606329 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.606335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.606517 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.606589 (+ 72us) client_negotiation.cc:769] Sending connection context
0812 07:11:45.606678 (+ 89us) client_negotiation.cc:240] Negotiation successful
0812 07:11:45.606862 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":139,"thread_start_us":537,"threads_started":1}
I20260812 07:11:45.607367 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.598357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:45913 (local address 127.21.181.66:57179)
0812 07:11:45.599194 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0812 07:11:45.599209 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:11:45.599315 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:11:45.599606 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:11:45.599610 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:11:45.599632 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:11:45.600535 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.600548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.601932 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.601936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.604611 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:11:45.604621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.606260 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.606268 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:11:45.606500 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.606585 (+ 85us) client_negotiation.cc:769] Sending connection context
0812 07:11:45.606678 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:11:45.606862 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":730,"mutex_wait_us":115,"thread_start_us":567,"threads_started":1}
I20260812 07:11:45.608407 11560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2f2282f32f48d48559c600a5c8c291" candidate_uuid: "785d7f0c6e6647bb819ac5923380ff59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d14fffc10f493e98d608cbfb696921" is_pre_election: true
I20260812 07:11:45.608408 11096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2f2282f32f48d48559c600a5c8c291" candidate_uuid: "785d7f0c6e6647bb819ac5923380ff59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e34fdede2ab54eb48593dcf1b13deb65" is_pre_election: true
I20260812 07:11:45.608871 11560 raft_consensus.cc:2454] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 785d7f0c6e6647bb819ac5923380ff59 in term 0.
I20260812 07:11:45.608916 11096 raft_consensus.cc:2454] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 785d7f0c6e6647bb819ac5923380ff59 in term 0.
I20260812 07:11:45.609570 11225 leader_election.cc:302] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73d14fffc10f493e98d608cbfb696921, 785d7f0c6e6647bb819ac5923380ff59; no voters:
I20260812 07:11:45.609946 11686 raft_consensus.cc:2781] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:11:45.610090 11686 raft_consensus.cc:482] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:11:45.610297 11686 raft_consensus.cc:3037] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:45.612903 11686 raft_consensus.cc:504] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.613865 11686 leader_election.cc:288] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [CANDIDATE]: Term 1 election: Requested vote from peers e34fdede2ab54eb48593dcf1b13deb65 (127.21.181.65:45729), 73d14fffc10f493e98d608cbfb696921 (127.21.181.67:45913)
I20260812 07:11:45.614141 11096 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2f2282f32f48d48559c600a5c8c291" candidate_uuid: "785d7f0c6e6647bb819ac5923380ff59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e34fdede2ab54eb48593dcf1b13deb65"
I20260812 07:11:45.614315 11096 raft_consensus.cc:3037] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:45.614292 11560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c2f2282f32f48d48559c600a5c8c291" candidate_uuid: "785d7f0c6e6647bb819ac5923380ff59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73d14fffc10f493e98d608cbfb696921"
I20260812 07:11:45.614466 11560 raft_consensus.cc:3037] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:11:45.617007 11560 raft_consensus.cc:2454] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 785d7f0c6e6647bb819ac5923380ff59 in term 1.
I20260812 07:11:45.617393 11096 raft_consensus.cc:2454] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 785d7f0c6e6647bb819ac5923380ff59 in term 1.
I20260812 07:11:45.617436 11225 leader_election.cc:302] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73d14fffc10f493e98d608cbfb696921, 785d7f0c6e6647bb819ac5923380ff59; no voters:
I20260812 07:11:45.617684 11686 raft_consensus.cc:2781] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:11:45.618147 11686 raft_consensus.cc:686] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 1 LEADER]: Becoming Leader. State: Replica: 785d7f0c6e6647bb819ac5923380ff59, State: Running, Role: LEADER
I20260812 07:11:45.618636 11686 consensus_queue.cc:237] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } }
I20260812 07:11:45.624215 10794 catalog_manager.cc:5495] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 785d7f0c6e6647bb819ac5923380ff59 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "785d7f0c6e6647bb819ac5923380ff59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "785d7f0c6e6647bb819ac5923380ff59" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39851 } health_report { overall_health: HEALTHY } } }
I20260812 07:11:45.646337 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.640621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36936 (local address 127.21.181.66:39851)
0812 07:11:45.640846 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.640851 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.640869 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:11:45.641046 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.641051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.641144 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.641266 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:11:45.641772 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.642581 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.644288 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.644678 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.644787 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.645306 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:11:45.645764 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:11:45.645893 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.645997 (+ 104us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.646076 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:11:45.654125 11560 raft_consensus.cc:1261] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 1 FOLLOWER]: Refusing update from remote peer 785d7f0c6e6647bb819ac5923380ff59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:45.654170 11096 raft_consensus.cc:1261] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 1 FOLLOWER]: Refusing update from remote peer 785d7f0c6e6647bb819ac5923380ff59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:11:45.655010 11698 consensus_queue.cc:1035] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73d14fffc10f493e98d608cbfb696921" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 45913 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:45.655354 11686 consensus_queue.cc:1035] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e34fdede2ab54eb48593dcf1b13deb65" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 45729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:11:45.679716 10794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:45.680008 10794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:11:45.683590 10794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 39310523fc574d98a5eaadea9e5cdd3f: Sending DeleteTablet for 3 replicas of tablet 8c2f2282f32f48d48559c600a5c8c291
I20260812 07:11:45.684615 11065 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c2f2282f32f48d48559c600a5c8c291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:45 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:39414
I20260812 07:11:45.684679 11532 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c2f2282f32f48d48559c600a5c8c291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:45 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:47080
I20260812 07:11:45.685073 11320 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c2f2282f32f48d48559c600a5c8c291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:11:45 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:36924
I20260812 07:11:45.687098 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 10944
I20260812 07:11:45.692001 11725 tablet_replica.cc:330] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: stopping tablet replica
I20260812 07:11:45.692749 11727 tablet_replica.cc:330] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: stopping tablet replica
I20260812 07:11:45.693027 11729 tablet_replica.cc:330] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: stopping tablet replica
I20260812 07:11:45.693135 11727 raft_consensus.cc:2229] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:45.693140 11725 raft_consensus.cc:2229] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:11:45.693392 11729 raft_consensus.cc:2229] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:11:45.693589 11725 raft_consensus.cc:2258] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:45.694036 11729 raft_consensus.cc:2258] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:45.694267 11727 raft_consensus.cc:2258] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:11:45.696470 11729 ts_tablet_manager.cc:1905] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:45.698197 11725 ts_tablet_manager.cc:1905] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:45.705060 11727 ts_tablet_manager.cc:1905] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:11:45.706096 11725 ts_tablet_manager.cc:1918] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:45.706283 11725 log.cc:1196] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/8c2f2282f32f48d48559c600a5c8c291
I20260812 07:11:45.706785 11725 ts_tablet_manager.cc:1939] T 8c2f2282f32f48d48559c600a5c8c291 P e34fdede2ab54eb48593dcf1b13deb65: Deleting consensus metadata
I20260812 07:11:45.708447 10772 catalog_manager.cc:4841] TS e34fdede2ab54eb48593dcf1b13deb65 (127.21.181.65:45729): tablet 8c2f2282f32f48d48559c600a5c8c291 (table test-table [id=6f0e398c54614139abbe0f266d1c0dd7]) successfully deleted
I20260812 07:11:45.709476 11729 ts_tablet_manager.cc:1918] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:45.709647 11729 log.cc:1196] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/8c2f2282f32f48d48559c600a5c8c291
I20260812 07:11:45.710177 11729 ts_tablet_manager.cc:1939] T 8c2f2282f32f48d48559c600a5c8c291 P 785d7f0c6e6647bb819ac5923380ff59: Deleting consensus metadata
I20260812 07:11:45.711853 10772 catalog_manager.cc:4841] TS 785d7f0c6e6647bb819ac5923380ff59 (127.21.181.66:39851): tablet 8c2f2282f32f48d48559c600a5c8c291 (table test-table [id=6f0e398c54614139abbe0f266d1c0dd7]) successfully deleted
I20260812 07:11:45.712855 11727 ts_tablet_manager.cc:1918] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:11:45.713022 11727 log.cc:1196] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/8c2f2282f32f48d48559c600a5c8c291
I20260812 07:11:45.713466 11727 ts_tablet_manager.cc:1939] T 8c2f2282f32f48d48559c600a5c8c291 P 73d14fffc10f493e98d608cbfb696921: Deleting consensus metadata
I20260812 07:11:45.715078 10773 catalog_manager.cc:4841] TS 73d14fffc10f493e98d608cbfb696921 (127.21.181.67:45913): tablet 8c2f2282f32f48d48559c600a5c8c291 (table test-table [id=6f0e398c54614139abbe0f266d1c0dd7]) successfully deleted
Aug 12 07:11:45 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:11:45.722220 11666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.688696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.21.181.65:45729)
0812 07:11:45.688845 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.688851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.704626 (+ 15775us) server_negotiation.cc:407] Connection header received
0812 07:11:45.705060 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.705065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.705148 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.705224 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:45.707091 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.708267 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.710421 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.711049 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.711152 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.716717 (+ 5565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:45.716728 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:45.716740 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:45.716781 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:45.719381 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.720066 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.720073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.720075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.720246 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.720599 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.720605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.720608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.720988 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:45.721313 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.721574 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.721812 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260812 07:11:45.722846 11166 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:11:45.811228 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 10944
I20260812 07:11:45.837882 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 11182
Aug 12 07:11:45 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:11:45.854848 11673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.838894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.21.181.66:39851)
0812 07:11:45.839116 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.839122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.839145 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:11:45.839214 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.839220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.839320 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.839453 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:45.840801 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.842081 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.843345 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.844010 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.844170 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:45.847984 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:45.848002 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:45.848026 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:45.848070 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:45.851947 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.852610 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.852619 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.852623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.852779 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:45.853102 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:45.853110 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:45.853114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:45.853594 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:45.853916 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:45.854124 (+ 208us) server_negotiation.cc:299] Negotiation successful
0812 07:11:45.854348 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:11:45.855527 11401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:11:45.966941 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 11182
I20260812 07:11:45.992929 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 11421
Aug 12 07:11:45 dist-test-slave-lgr5 krb5kdc[10012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:11:46.007411 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:11:45.993746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.21.181.67:45913)
0812 07:11:45.993932 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:11:45.993936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:11:45.993987 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:11:45.994167 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:11:45.994171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:11:45.994241 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:11:45.994328 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:11:45.995546 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.996336 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.997683 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:11:45.998077 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:11:45.998173 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:11:46.002291 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:11:46.002304 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:11:46.002343 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:11:46.002378 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:11:46.005079 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:46.005697 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:46.005703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:46.005706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:46.005846 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:11:46.006129 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:11:46.006132 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:11:46.006134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:11:46.006494 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:11:46.006713 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:11:46.006903 (+ 190us) server_negotiation.cc:299] Negotiation successful
0812 07:11:46.007055 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:11:46.008034 11632 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:11:46.093513 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 11421
I20260812 07:11:46.121037 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 10740
I20260812 07:11:46.122231 10853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:11:46.233541 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 10740
I20260812 07:11:46.267635 22229 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 07:11:46.611665 22229 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 07:11:46.611876 22229 mini_ranger.cc:66] Stopping Ranger...
I20260812 07:11:46.955847 22229 mini_ranger.cc:68] Stopped Ranger
2026-08-12 07:11:46.956 UTC [10054] LOG: received smart shutdown request
2026-08-12 07:11:46.959 UTC [10054] LOG: background worker "logical replication launcher" (PID 10075) exited with exit code 1
2026-08-12 07:11:46.959 UTC [10059] LOG: shutting down
2026-08-12 07:11:47.081 UTC [10054] LOG: database system is shut down
2026-08-12T07:11:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47801 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:11:47 dist-test-slave-lgr5 krb5kdc[11763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:11:47 dist-test-slave-lgr5 krb5kdc[11763](info): set up 2 sockets
Aug 12 07:11:47 dist-test-slave-lgr5 krb5kdc[11763](info): commencing operation
krb5kdc: starting...
W20260812 07:11:48.435519 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.001s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:11:48 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:11:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:11:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/rangeradmin_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/rangeradmin_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/rangerlookup_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/rangerlookup_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
I20260812 07:11:48.747833 22229 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCYQ_m9/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/postgres -l logfile start
2026-08-12 07:11:50.914 UTC [11799] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-12 07:11:50.914 UTC [11799] LOG: listening on IPv4 address "127.21.181.84", port 45275
2026-08-12 07:11:50.917 UTC [11799] LOG: listening on Unix socket "/tmp/.s.PGSQL.45275"
2026-08-12 07:11:50.938 UTC [11801] LOG: database system was shut down at 2026-08-12 07:11:49 UTC
2026-08-12 07:11:50.942 UTC [11799] LOG: database system is ready to accept connections
I20260812 07:11:52.088482 22229 mini_postgres.cc:96] Postgres bound to 45275
2026-08-12 07:11:52.111 UTC [11811] FATAL: database "slave" does not exist
127.21.181.84:45275 - accepting connections
I20260812 07:11:52.111691 22229 mini_ranger.cc:161] Starting Ranger...
I20260812 07:11:52.139204 22229 mini_ranger.cc:84] Created miniranger Postgres user
I20260812 07:11:52.524183 22229 mini_ranger.cc:87] Created ranger Postgres database
I20260812 07:11:52.524281 22229 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-admin
2026-08-12 07:11:52,794 [I] DB FLAVOR :POSTGRES
2026-08-12 07:11:52,795 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:11:52,795 [I] Checking connection..
2026-08-12 07:11:52,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-12 07:11:53,050 [I] Checking connection passed.
2026-08-12 07:11:53,051 [I] --------- Verifying version history table ---------
2026-08-12 07:11:53,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:11:53,290 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:11:53,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:11:53,537 [I] Table x_db_version_h does not exist in database ranger
2026-08-12 07:11:53,538 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-12 07:11:53,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-12 07:11:53.800 UTC [11920] WARNING: there is no transaction in progress
2026-08-12 07:11:53,809 [I] create_dbversion_catalog.sql file imported successfully
2026-08-12 07:11:53,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-12 07:11:54,071 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-12 07:11:54,071 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:11:54,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-12 07:11:54,329 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-12 07:11:54,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-lgr5.c.gcp-upstream.internal','N') ;"
2026-08-12 07:11:54,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:11:55,071 [I] Table x_portal_user does not exist in database ranger
2026-08-12 07:11:55,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:11:55,326 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-12 07:11:55,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:11:55,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:11:55,798 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-12 07:11:55,798 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-12 07:11:56.653 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.668 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.683 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.695 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.711 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.726 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.773 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.779 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.790 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.799 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.808 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.820 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.828 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.833 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.839 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:56.844 UTC [12091] WARNING: there is no transaction in progress
2026-08-12 07:11:57,347 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-12 07:11:57,347 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-12 07:11:57,592 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-12 07:11:57,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-12 07:11:57,845 [I] Table x_portal_user already exists in database 'ranger'
2026-08-12 07:11:57,845 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-12 07:11:58,100 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-12 07:11:58,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-12 07:11:58,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-12 07:11:58,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-lgr5.c.gcp-upstream.internal';"
2026-08-12 07:11:58,819 [I] CORE_DB_SCHEMA import status has been updated
2026-08-12 07:11:58,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-12 07:11:59,078 [I] DB_PATCHES have already been applied
I20260812 07:11:59.084470 22229 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCYQ_m9/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260812 07:11:59.084563 22229 mini_ranger.cc:193] Using host: 127.21.181.84
I20260812 07:11:59.103976 22229 mini_ranger.cc:227] Ranger admin URL: http://127.21.181.84:44445
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-admin/ews/webapp].
Aug 12, 2026 7:11:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260812 07:12:00.381708 22229 mini_ranger.cc:160] Time spent starting Ranger: real 8.270s user 0.007s sys 0.044s
Aug 12, 2026 7:12:19 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 12, 2026 7:12:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 12, 2026 7:12:19 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 12, 2026 7:12:19 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 12, 2026 7:12:20 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260812 07:12:21.429131 22229 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.21.181.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.21.181.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/rangerkms_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.21.181.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/rangerkms_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.21.181.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.21.181.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.21.181.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/HTTP_127.21.181.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 12 07:12:21 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518741, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260812 07:12:21.602852 22229 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260812 07:12:21.625833 22229 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260812 07:12:22.312924 22229 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260812 07:12:22.313026 22229 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms
I20260812 07:12:22.313071 22229 mini_ranger_kms.cc:227] Using postgres at 127.21.181.84:45275
2026-08-12 07:12:22,665 [I] DB FLAVOR :POSTGRES
2026-08-12 07:12:22,665 [I] --------- Verifying Ranger DB connection ---------
2026-08-12 07:12:22,665 [I] Checking connection
2026-08-12 07:12:22,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-12 07:12:22,915 [I] connection success
2026-08-12 07:12:22,915 [I] --------- Verifying Ranger DB tables ---------
2026-08-12 07:12:22,915 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-12 07:12:22,915 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-12 07:12:23,137 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-12 07:12:23,137 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-12 07:12:23,138 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-12 07:12:23,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.21.181.84:45275/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-12 07:12:23,381 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260812 07:12:23.786521 22229 mini_ranger_kms.cc:310] Created kms service
I20260812 07:12:23.986943 22229 mini_ranger_kms.cc:326] Created kudu user
I20260812 07:12:24.035808 22229 mini_ranger_kms.cc:343] Created rangerkms user
Aug 12 07:12:24 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518741, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:24.426134 22229 mini_ranger_kms.cc:384] Added ranger policy
I20260812 07:12:24.426460 22229 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCYQ_m9/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCYQ_m9/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260812 07:12:24.426517 22229 mini_ranger_kms.cc:240] Using host: 127.21.181.84
I20260812 07:12:24.447291 22229 mini_ranger_kms.cc:276] Ranger KMS PID: 12388
I20260812 07:12:24.447422 22229 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.21.181.84:35121
Aug 12, 2026 7:12:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 12, 2026 7:12:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 12, 2026 7:12:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 12, 2026 7:12:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 12, 2026 7:12:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-kms/ews/webapp].
Aug 12, 2026 7:12:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 12 07:12:26 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518746, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.181.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:27.043043 22229 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.440s user 0.084s sys 0.297s
I20260812 07:12:27.043295 22229 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260812 07:12:27.043464 22229 mini_ranger_kms.cc:401] 127.21.181.84:35121/kms/v1/keys
Aug 12 07:12:27 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518746, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.181.84@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
Aug 12 07:12:27 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518746, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.21.181.84@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
Aug 12 07:12:29 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:12:29.613747 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45651
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:45651
--ranger_config_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:29.951686 12469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:29.951960 12469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:29.952103 12469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:29.959115 12469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:29.959218 12469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:29.959292 12469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:29.959368 12469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:29.959434 12469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:29.969089 12469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
--ranger_kms_url=127.21.181.84:35121/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:45651
--ranger_config_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:29.970292 12469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:29.971684 12469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:29.983637 12480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:29.983671 12483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:29.983696 12481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:29.984620 12469 server_base.cc:1034] running on GCE node
I20260812 07:12:29.985306 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:29.986569 12469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:29.987764 12469 hybrid_clock.cc:648] HybridClock initialized: now 1786518749987713 us; error 43 us; skew 500 ppm
I20260812 07:12:29.988109 12469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:29 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:29.990983 12469 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:29.992404 12469 webserver.cc:460] Webserver started at http://127.21.181.126:34021/ using document root <none> and password file <none>
I20260812 07:12:29.993000 12469 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:29.993110 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:29.993393 12469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:12:29 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:30.051614 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "53b267ad1d774f53829f0b152d739cb6"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-lgr5"
server_key: "48c0be35729613a0065e5ea3dcfecca0"
server_key_iv: "8ccfe1809f915dd14a3480854b565008"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:30.052615 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "53b267ad1d774f53829f0b152d739cb6"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-lgr5"
server_key: "48c0be35729613a0065e5ea3dcfecca0"
server_key_iv: "8ccfe1809f915dd14a3480854b565008"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:30.057876 12469 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:12:30.073614 12495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:30.075371 12469 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:30.075567 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "53b267ad1d774f53829f0b152d739cb6"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-lgr5"
server_key: "48c0be35729613a0065e5ea3dcfecca0"
server_key_iv: "8ccfe1809f915dd14a3480854b565008"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:30.075819 12469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:30.088470 12469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:30.092223 12469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:30.092553 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:30.109664 12469 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:45651
I20260812 07:12:30.109684 12579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:45651 every 8 connection(s)
I20260812 07:12:30.110896 12469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:30.113436 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 12469
I20260812 07:12:30.113646 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:12:30.114495 22229 external_mini_cluster.cc:1428] Setting key 62ea941f58bc398a2c747489f6d4e68a
I20260812 07:12:30.115000 12583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:30.128551 12583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Bootstrap starting.
Aug 12 07:12:30 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:30.131462 12583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:30.132524 12583 log.cc:824] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:30.135057 12583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: No bootstrap required, opened a new log
I20260812 07:12:30.136425 12588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:30.116480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.21.181.126:45651)
0812 07:12:30.117240 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:30.117251 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:30.117290 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:12:30.119400 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:30.119422 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:30.119766 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:30.120126 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:30.121628 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.122858 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.124652 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:30.125313 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:30.125451 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:30.129752 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:30.129762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:30.129775 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:30.129814 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:30.132607 (+ 2793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:30.133230 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:30.133237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:30.133243 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:30.133402 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:30.133763 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:30.133766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:30.133769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:30.134240 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:30.134541 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:30.134938 (+ 397us) server_negotiation.cc:299] Negotiation successful
0812 07:12:30.135435 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":355,"threads_started":1}
I20260812 07:12:30.142316 12583 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:30.142656 12583 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:30.142743 12583 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b267ad1d774f53829f0b152d739cb6, State: Initialized, Role: FOLLOWER
I20260812 07:12:30.143260 12583 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:30.143492 12583 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:30.143612 12583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:30.143759 12583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:30.145802 12583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:30.146242 12583 leader_election.cc:302] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b267ad1d774f53829f0b152d739cb6; no voters:
I20260812 07:12:30.146759 12583 leader_election.cc:288] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:30.146862 12590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:30.147468 12590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 LEADER]: Becoming Leader. State: Replica: 53b267ad1d774f53829f0b152d739cb6, State: Running, Role: LEADER
I20260812 07:12:30.147954 12590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:30.148623 12583 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:30.150383 12592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53b267ad1d774f53829f0b152d739cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } } }
I20260812 07:12:30.150700 12592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:30.152176 12593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b267ad1d774f53829f0b152d739cb6. Latest consensus state: current_term: 1 leader_uuid: "53b267ad1d774f53829f0b152d739cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } } }
I20260812 07:12:30.152432 12593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:30.152701 12601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:30.157438 12601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:30.159749 12583 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:12:30.174894 12601 catalog_manager.cc:1347] Generated new cluster ID: 70d6adb195b7463bb1e49bb65efee8cb
I20260812 07:12:30.175033 12601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:30.196601 12601 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:30.198422 12601 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:30.212375 12601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Generated new TSK 0
I20260812 07:12:30.213444 12601 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:12:30.967914 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45651
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:12:31 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260812 07:12:31.420045 12669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:31.420486 12669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:31.420694 12669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:31.432045 12669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:31.432231 12669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:31.432464 12669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:12:31.447086 12669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.21.181.84:35121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:31.448390 12669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:31.449764 12669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:31.464097 12689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:31.464128 12690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:31.464865 12693 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:31.466033 12669 server_base.cc:1034] running on GCE node
I20260812 07:12:31.466656 12669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:31.467581 12669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:31.468776 12669 hybrid_clock.cc:648] HybridClock initialized: now 1786518751468713 us; error 59 us; skew 500 ppm
I20260812 07:12:31.469087 12669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:31 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:31.472167 12669 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:12:31.485303 12669 webserver.cc:460] Webserver started at http://127.21.181.65:38539/ using document root <none> and password file <none>
I20260812 07:12:31.486071 12669 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:31.486193 12669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:31.486516 12669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:12:31 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:31.510100 12669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "8511d54da6234bfba9b66c8488a8c857"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-lgr5"
server_key: "c5db433bb1825225f66ea5dfe59696f9"
server_key_iv: "ec1b7007d49fc06648521ab14f7c8e2f"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:31.511344 12669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "8511d54da6234bfba9b66c8488a8c857"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-lgr5"
server_key: "c5db433bb1825225f66ea5dfe59696f9"
server_key_iv: "ec1b7007d49fc06648521ab14f7c8e2f"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:31.520345 12669 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260812 07:12:31.543107 12706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.544760 12669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:31.544950 12669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "8511d54da6234bfba9b66c8488a8c857"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-lgr5"
server_key: "c5db433bb1825225f66ea5dfe59696f9"
server_key_iv: "ec1b7007d49fc06648521ab14f7c8e2f"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:31.545182 12669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:31.567148 12669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:31.572156 12669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:31.572500 12669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:31.573647 12669 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:31.575219 12669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:31.575354 12669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.575517 12669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:31.575608 12669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:31.624125 12669 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:35545
I20260812 07:12:31.624177 12877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:35545 every 8 connection(s)
I20260812 07:12:31.625545 12669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:31.632392 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 12669
I20260812 07:12:31.632623 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:31.633570 22229 external_mini_cluster.cc:1428] Setting key eff169119ba8780fdc448ff5cfbcbcd3
Aug 12 07:12:31 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
Aug 12 07:12:31 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:31.698523 12883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.631293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45651 (local address 127.21.181.65:44153)
0812 07:12:31.668105 (+ 36812us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:31.668159 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:31.670172 (+ 2013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:31.674525 (+ 4353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:31.674538 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:31.675213 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:31.676112 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.676143 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.677857 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.677862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.679884 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:31.679912 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.684983 (+ 5071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.684992 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:31.685298 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.686094 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:31.686127 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:31.689545 (+ 3418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:31.693282 (+ 3737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:31.693292 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:31.693310 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:31.693958 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:31.694498 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:31.694503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:31.694507 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:31.694814 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:31.695646 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:31.695654 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:31.696160 (+ 506us) client_negotiation.cc:769] Sending connection context
0812 07:12:31.696515 (+ 355us) client_negotiation.cc:240] Negotiation successful
0812 07:12:31.697072 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36462,"thread_start_us":36207,"threads_started":1}
I20260812 07:12:31.699752 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:31.630631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:44153 (local address 127.21.181.126:45651)
0812 07:12:31.673876 (+ 43245us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:31.673884 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:31.673909 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:12:31.673984 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:31.673990 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:31.674085 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:31.674221 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:31.676335 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.677643 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.680094 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:31.680859 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:31.680982 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:31.689785 (+ 8803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:31.689791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:31.689794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:31.689836 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:31.693111 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.694146 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.694152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.694156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.694331 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:31.694979 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:31.694985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:31.694988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:31.695401 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:31.695547 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:31.699086 (+ 3539us) server_negotiation.cc:299] Negotiation successful
0812 07:12:31.699328 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43090,"thread_start_us":43742,"threads_started":1}
I20260812 07:12:31.703531 12879 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45651
I20260812 07:12:31.703873 12879 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:31.704849 12879 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
I20260812 07:12:31.707500 12527 ts_manager.cc:194] Registered new tserver with Master: 8511d54da6234bfba9b66c8488a8c857 (127.21.181.65:35545)
I20260812 07:12:31.709643 12527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:44153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:12:31.747488 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45651
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:12:31 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518751, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
W20260812 07:12:32.197901 12887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:32.198252 12887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:32.198408 12887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:32.207459 12887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:32.207574 12887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:32.207751 12887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:12:32.217576 12887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.21.181.84:35121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:32.218839 12887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:32.220140 12887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:32.235924 12901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.235930 12902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.244654 12887 server_base.cc:1034] running on GCE node
W20260812 07:12:32.245220 12904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.245922 12887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:32.254647 12887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:32.262125 12887 hybrid_clock.cc:648] HybridClock initialized: now 1786518752261676 us; error 431 us; skew 500 ppm
I20260812 07:12:32.262557 12887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:32.266613 12887 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:12:32.268226 12887 webserver.cc:460] Webserver started at http://127.21.181.66:39507/ using document root <none> and password file <none>
I20260812 07:12:32.268934 12887 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:32.269052 12887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:32.269351 12887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:32.286043 12887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "d439871a6fb34bf0bd950b06c039ec48"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "5d3926d3fd1bbb69870fdd362decda19"
server_key_iv: "b1d805431c05ef797bc4b25704ebdb18"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:32.287060 12887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "d439871a6fb34bf0bd950b06c039ec48"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "5d3926d3fd1bbb69870fdd362decda19"
server_key_iv: "b1d805431c05ef797bc4b25704ebdb18"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:32.292424 12887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260812 07:12:32.308466 12925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.309854 12887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:12:32.310005 12887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "d439871a6fb34bf0bd950b06c039ec48"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "5d3926d3fd1bbb69870fdd362decda19"
server_key_iv: "b1d805431c05ef797bc4b25704ebdb18"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:32.310199 12887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:32.329339 12887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:32.333163 12887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:32.333441 12887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:32.334403 12887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:32.335893 12887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:32.335973 12887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.336120 12887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:32.336194 12887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.371675 12887 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:37791
I20260812 07:12:32.371692 13110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:37791 every 8 connection(s)
I20260812 07:12:32.373046 12887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:32.379623 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 12887
I20260812 07:12:32.379796 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:32.380532 22229 external_mini_cluster.cc:1428] Setting key 77130cf9d7319143ad25f71c07c6f033
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:32.444154 13120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.378469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45651 (local address 127.21.181.66:40805)
0812 07:12:32.413410 (+ 34941us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:32.413484 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:32.422551 (+ 9067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:32.423898 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:32.423913 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:32.425908 (+ 1995us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:32.426518 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:32.426536 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.428728 (+ 2192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.428733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:32.430625 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:32.430641 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.431754 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.431761 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:32.432066 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.432825 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:32.432854 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:32.435440 (+ 2586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:32.439150 (+ 3710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:32.439158 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:32.439174 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:32.439778 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:32.440221 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:32.440226 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:32.440229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:32.440443 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:32.441234 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:32.441242 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:32.441692 (+ 450us) client_negotiation.cc:769] Sending connection context
0812 07:12:32.442002 (+ 310us) client_negotiation.cc:240] Negotiation successful
0812 07:12:32.442546 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34512,"thread_start_us":35058,"threads_started":1}
I20260812 07:12:32.445261 13116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.377468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:40805 (local address 127.21.181.126:45651)
0812 07:12:32.419524 (+ 42056us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:32.419532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:32.419564 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:12:32.423107 (+ 3543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:32.423115 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:32.423229 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:32.423332 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:32.426703 (+ 3371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.428538 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.430802 (+ 2264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.431331 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.431462 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.435690 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:32.435696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:32.435700 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:32.435738 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:32.438973 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:32.439916 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:32.439922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:32.439926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:32.440073 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:32.440586 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:32.440592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:32.440595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:32.440993 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:32.441132 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:32.444569 (+ 3437us) server_negotiation.cc:299] Negotiation successful
0812 07:12:32.444878 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41884,"thread_start_us":45968,"threads_started":1}
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
I20260812 07:12:32.447240 13111 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45651
I20260812 07:12:32.447548 13111 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:32.448293 13111 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
I20260812 07:12:32.449920 12527 ts_manager.cc:194] Registered new tserver with Master: d439871a6fb34bf0bd950b06c039ec48 (127.21.181.66:37791)
I20260812 07:12:32.450709 12527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:40805
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:12:32.481500 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45651
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:32.713655 12879 heartbeater.cc:503] Master 127.21.181.126:45651 was elected leader, sending a full tablet report...
W20260812 07:12:32.812461 13124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:32.812700 13124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:32.812830 13124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:32.819829 13124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:32.819927 13124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:32.820072 13124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:12:32.829994 13124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.21.181.84:35121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:32.831144 13124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:32.832432 13124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:32.844811 13133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.844892 13132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:32.844859 13137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:32.846067 13124 server_base.cc:1034] running on GCE node
I20260812 07:12:32.846630 13124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:32.847571 13124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:32.848763 13124 hybrid_clock.cc:648] HybridClock initialized: now 1786518752848720 us; error 32 us; skew 500 ppm
I20260812 07:12:32.849105 13124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:32.852264 13124 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:12:32.853647 13124 webserver.cc:460] Webserver started at http://127.21.181.67:33535/ using document root <none> and password file <none>
I20260812 07:12:32.854276 13124 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:32.854390 13124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:32.854678 13124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:32.867756 13124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "53e13b407eaa4035bb7e126015c87567"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "27a5decb074cf2328c64ab9961607e1c"
server_key_iv: "1bbc30ba6fda5e1bb7d5a7eaf1831b42"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:32.868582 13124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "53e13b407eaa4035bb7e126015c87567"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "27a5decb074cf2328c64ab9961607e1c"
server_key_iv: "1bbc30ba6fda5e1bb7d5a7eaf1831b42"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:32.873697 13124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260812 07:12:32.890107 13150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.891741 13124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:32.891912 13124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "53e13b407eaa4035bb7e126015c87567"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "27a5decb074cf2328c64ab9961607e1c"
server_key_iv: "1bbc30ba6fda5e1bb7d5a7eaf1831b42"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:32.892211 13124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:32.914678 13124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:32.918195 13124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:32.918435 13124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:32.919260 13124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:32.920550 13124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:32.920626 13124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.920717 13124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:32.920758 13124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:32.950591 13124 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:34025
I20260812 07:12:32.950637 13353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:34025 every 8 connection(s)
I20260812 07:12:32.951795 13124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:32.957681 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 13124
I20260812 07:12:32.957842 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:32.958348 22229 external_mini_cluster.cc:1428] Setting key 0d8ff4e12d66d818a64e81b34b4a5436
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518752, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:32.970158 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.954162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:37643 (local address 127.21.181.126:45651)
0812 07:12:32.954662 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:32.954669 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:32.955580 (+ 911us) server_negotiation.cc:407] Connection header received
0812 07:12:32.957334 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:32.957339 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:32.957442 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:32.957575 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:32.958961 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.959811 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.961376 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.961815 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.961926 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.964932 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:32.964939 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:32.964943 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:32.964986 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:32.967303 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:32.967948 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:32.967952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:32.967954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:32.968113 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:32.968545 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:32.968551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:32.968554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:32.968812 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:32.968951 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:32.969588 (+ 637us) server_negotiation.cc:299] Negotiation successful
0812 07:12:32.969815 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":223,"threads_started":1}
I20260812 07:12:32.970939 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.954746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45651 (local address 127.21.181.67:37643)
0812 07:12:32.955428 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:32.955472 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:32.957102 (+ 1630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:32.957735 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:32.957742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:32.958221 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:32.958795 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:32.958812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.959966 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.959970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:32.961230 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:32.961251 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.962042 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.962050 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:32.962259 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.962798 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:32.962819 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:32.964732 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:32.967430 (+ 2698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:32.967439 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:32.967452 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:32.967824 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:32.968219 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:32.968224 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:32.968228 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:32.968421 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:32.968937 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:32.968944 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:32.969233 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 07:12:32.969448 (+ 215us) client_negotiation.cc:240] Negotiation successful
0812 07:12:32.969884 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":279,"threads_started":1}
I20260812 07:12:32.972273 13355 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45651
I20260812 07:12:32.972568 13355 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:32.973261 13355 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
I20260812 07:12:32.974567 12527 ts_manager.cc:194] Registered new tserver with Master: 53e13b407eaa4035bb7e126015c87567 (127.21.181.67:34025)
I20260812 07:12:32.975358 12527 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:37643
I20260812 07:12:32.984903 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:32.986534 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 12669
Aug 12 07:12:32 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:12:33.001614 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:32.987411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.21.181.65:35545)
0812 07:12:32.988004 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:32.988010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:32.988032 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:12:32.988121 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:32.988130 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:32.988308 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:32.988422 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:32.989727 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.990697 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.992043 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:32.992593 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:32.992694 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:32.996590 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:32.996600 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:32.996611 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:32.996641 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:32.999279 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:32.999900 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:32.999906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:32.999908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:33.000061 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:33.000400 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:33.000403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:33.000405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:33.000723 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:33.000940 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.001137 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.001326 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":309,"threads_started":1}
I20260812 07:12:33.002806 12872 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:33.084002 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 12669
I20260812 07:12:33.107898 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 12887
Aug 12 07:12:33 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:12:33.123991 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.108850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45474 (local address 127.21.181.66:37791)
0812 07:12:33.109399 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.109408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.109440 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:12:33.109510 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.109518 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.109796 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.109973 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:33.111144 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.112724 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.114079 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.114952 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.115117 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.119160 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:33.119172 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:33.119185 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:33.119217 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:33.121716 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:33.122304 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:33.122311 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:33.122313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:33.122436 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:33.122692 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:33.122695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:33.122697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:33.123025 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:33.123249 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.123424 (+ 175us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.123620 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":261,"threads_started":1}
I20260812 07:12:33.125187 13106 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:33.204977 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 12887
I20260812 07:12:33.223730 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 13124
Aug 12 07:12:33 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:12:33.239221 13369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:33.224673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45304 (local address 127.21.181.67:34025)
0812 07:12:33.225284 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:33.225291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:33.225321 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:12:33.225381 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:33.225388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:33.225606 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:33.225734 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:33.227061 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.228120 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.229312 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:33.229861 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:33.229955 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:33.234105 (+ 4150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:33.234118 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:33.234139 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:33.234179 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:33.237014 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:33.237585 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:33.237592 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:33.237594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:33.237713 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:33.237995 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:33.237998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:33.238000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:33.238321 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:33.238557 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:33.238741 (+ 184us) server_negotiation.cc:299] Negotiation successful
0812 07:12:33.238930 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":294,"threads_started":1}
I20260812 07:12:33.240483 13344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:33.321283 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 13124
I20260812 07:12:33.348539 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 12469
I20260812 07:12:33.349879 12575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:33.454164 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 12469
I20260812 07:12:33.483937 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45651
--webserver_interface=127.21.181.126
--webserver_port=34021
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:45651
--ranger_config_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:33.797588 13375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:33.797852 13375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:33.797945 13375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:33.804814 13375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:33.804903 13375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:33.804957 13375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:33.805011 13375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:33.805058 13375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:33.814364 13375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:45651
--ranger_config_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=34021
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:33.815449 13375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:33.816681 13375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:33.827582 13382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:33.827713 13381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:33.827967 13385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:33.828341 13375 server_base.cc:1034] running on GCE node
I20260812 07:12:33.829159 13375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:33.830391 13375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:33.831590 13375 hybrid_clock.cc:648] HybridClock initialized: now 1786518753831533 us; error 44 us; skew 500 ppm
I20260812 07:12:33.831902 13375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:33 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518753, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:33.834785 13375 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:33.836153 13375 webserver.cc:460] Webserver started at http://127.21.181.126:34021/ using document root <none> and password file <none>
I20260812 07:12:33.836751 13375 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:33.836870 13375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:12:33 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518753, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:33.911443 13375 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.003s
I20260812 07:12:33.914537 13397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:33.915990 13375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:33.916182 13375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "53b267ad1d774f53829f0b152d739cb6"
format_stamp: "Formatted at 2026-08-12 07:12:30 on dist-test-slave-lgr5"
server_key: "48c0be35729613a0065e5ea3dcfecca0"
server_key_iv: "8ccfe1809f915dd14a3480854b565008"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:33.917157 13375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:33.930146 13375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:33.933812 13375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:33.934062 13375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:33.950727 13375 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:45651
I20260812 07:12:33.950750 13484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:45651 every 8 connection(s)
I20260812 07:12:33.952051 13375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:33.957810 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 13375
I20260812 07:12:33.958868 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:35545
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:45651
--webserver_port=38539
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:33.958985 13485 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:12:33.963097 13485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Bootstrap starting.
I20260812 07:12:33.985304 13485 log.cc:824] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:33.994200 13485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:12:33.994711 13485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Bootstrap complete.
I20260812 07:12:34.003647 13485 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:34.004508 13485 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b267ad1d774f53829f0b152d739cb6, State: Initialized, Role: FOLLOWER
I20260812 07:12:34.005082 13485 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:34.005296 13485 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:34.005414 13485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:34.005573 13485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:12:34.009244 13485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:34.009739 13485 leader_election.cc:302] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53b267ad1d774f53829f0b152d739cb6; no voters:
I20260812 07:12:34.010311 13485 leader_election.cc:288] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:12:34.010476 13492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:12:34.011548 13492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [term 2 LEADER]: Becoming Leader. State: Replica: 53b267ad1d774f53829f0b152d739cb6, State: Running, Role: LEADER
I20260812 07:12:34.012180 13492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } }
I20260812 07:12:34.012450 13485 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:34.016670 13494 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "53b267ad1d774f53829f0b152d739cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } } }
I20260812 07:12:34.017143 13494 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:34.017733 13495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53b267ad1d774f53829f0b152d739cb6. Latest consensus state: current_term: 2 leader_uuid: "53b267ad1d774f53829f0b152d739cb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53b267ad1d774f53829f0b152d739cb6" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45651 } } }
I20260812 07:12:34.018036 13495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:34.027894 13513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:34.033515 13513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:34.034301 13485 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260812 07:12:34.034924 13513 catalog_manager.cc:1259] Loaded cluster ID: 70d6adb195b7463bb1e49bb65efee8cb
I20260812 07:12:34.035017 13513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:34.039034 13513 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:34.040495 13513 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Loaded TSK: 0
I20260812 07:12:34.042081 13513 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:12:34.397387 13489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:34.397866 13489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:34.398070 13489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:34.406847 13489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:34.407019 13489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:34.407238 13489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:12:34.422073 13489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:35545
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=38539
--tserver_master_addrs=127.21.181.126:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:34.424294 13489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:34.426092 13489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:34.444861 13554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:34.448441 13558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:34.452137 13561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:34.452936 13489 server_base.cc:1034] running on GCE node
I20260812 07:12:34.453840 13489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:34.458534 13489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:34.459885 13489 hybrid_clock.cc:648] HybridClock initialized: now 1786518754459821 us; error 58 us; skew 500 ppm
I20260812 07:12:34.460255 13489 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:34 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518754, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:34.467041 13489 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:12:34.469347 13489 webserver.cc:460] Webserver started at http://127.21.181.65:38539/ using document root <none> and password file <none>
I20260812 07:12:34.470031 13489 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:34.470167 13489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:12:34 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518754, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:34.493666 13489 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:12:34.497085 13572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:34.498684 13489 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260812 07:12:34.498930 13489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "8511d54da6234bfba9b66c8488a8c857"
format_stamp: "Formatted at 2026-08-12 07:12:31 on dist-test-slave-lgr5"
server_key: "c5db433bb1825225f66ea5dfe59696f9"
server_key_iv: "ec1b7007d49fc06648521ab14f7c8e2f"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:34.500470 13489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:34.519105 13489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:34.524749 13489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:34.525146 13489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:34.530525 13489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:34.531986 13489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:34.532109 13489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:34.532220 13489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:34.532298 13489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:34.628407 13489 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:35545
I20260812 07:12:34.628489 13764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:35545 every 8 connection(s)
I20260812 07:12:34.630261 13489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:34.635263 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 13489
I20260812 07:12:34.636053 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:37791
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:45651
--webserver_port=39507
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:12:34 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518754, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:34.683223 13769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:34.635587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45651 (local address 127.21.181.65:43341)
0812 07:12:34.650325 (+ 14738us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:34.650395 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:34.652618 (+ 2223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:34.654443 (+ 1825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:34.654454 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:34.654952 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:34.655653 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:34.655682 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:34.657921 (+ 2239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:34.657927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:34.659819 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:34.659843 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:34.661206 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:34.661214 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:34.661494 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:34.664285 (+ 2791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:34.664330 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:34.667106 (+ 2776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:34.671496 (+ 4390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:34.671506 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:34.671522 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:34.672125 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:34.679104 (+ 6979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:34.679109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:34.679113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:34.679421 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:34.680506 (+ 1085us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:34.680514 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:34.680932 (+ 418us) client_negotiation.cc:769] Sending connection context
0812 07:12:34.681256 (+ 324us) client_negotiation.cc:240] Negotiation successful
0812 07:12:34.681831 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14453,"thread_start_us":14261,"threads_started":1}
I20260812 07:12:34.683393 13768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:34.635480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:43341 (local address 127.21.181.126:45651)
0812 07:12:34.652532 (+ 17052us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:34.652550 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:34.652603 (+ 53us) server_negotiation.cc:407] Connection header received
0812 07:12:34.653361 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:34.653382 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:34.653868 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:34.654264 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:34.655895 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:34.657744 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:34.659980 (+ 2236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:34.660978 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:34.661145 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:34.667406 (+ 6261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:34.667416 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:34.667435 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:34.667484 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:34.671294 (+ 3810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:34.672311 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:34.672333 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:34.672341 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:34.678940 (+ 6599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:34.679573 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:34.679579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:34.679583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:34.680322 (+ 739us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:34.680544 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:34.681567 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0812 07:12:34.681972 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16811,"thread_start_us":16602,"threads_started":1}
I20260812 07:12:34.686975 13765 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45651
I20260812 07:12:34.687325 13765 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:34.688215 13765 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
W20260812 07:12:34.691423 13765 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:45651 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:12:34.691660 13765 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:34.691951 13765 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
I20260812 07:12:34.694360 13765 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:34.694635 13765 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
W20260812 07:12:34.695809 13765 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:12:35.048753 13771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:35.049014 13771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:35.049160 13771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:35.056478 13771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:35.056586 13771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:35.056743 13771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:12:35.066816 13771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:37791
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=39507
--tserver_master_addrs=127.21.181.126:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:35.068053 13771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:35.069298 13771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:35.083492 13792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.084483 13795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.084511 13791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:35.085623 13771 server_base.cc:1034] running on GCE node
I20260812 07:12:35.086360 13771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:35.087798 13771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:35.089042 13771 hybrid_clock.cc:648] HybridClock initialized: now 1786518755088971 us; error 48 us; skew 500 ppm
I20260812 07:12:35.089397 13771 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:35.093078 13771 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:12:35.100395 13771 webserver.cc:460] Webserver started at http://127.21.181.66:39507/ using document root <none> and password file <none>
I20260812 07:12:35.101070 13771 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:35.101202 13771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:35.131486 13771 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.006s sys 0.004s
I20260812 07:12:35.134284 13820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.135660 13771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:35.135819 13771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "d439871a6fb34bf0bd950b06c039ec48"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "5d3926d3fd1bbb69870fdd362decda19"
server_key_iv: "b1d805431c05ef797bc4b25704ebdb18"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:35.136855 13771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:35.185366 13771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:35.190482 13771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:35.190855 13771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:35.192004 13771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:35.207985 13771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:35.208181 13771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260812 07:12:35.208374 13771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:35.208493 13771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.260192 13771 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:37791
I20260812 07:12:35.260277 14004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:37791 every 8 connection(s)
I20260812 07:12:35.262180 13771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:35.272531 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 13771
I20260812 07:12:35.273366 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.21.181.84:35121/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:34025
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:45651
--webserver_port=33535
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:35.296227 14008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.265778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:46845 (local address 127.21.181.126:45651)
0812 07:12:35.275419 (+ 9641us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:35.275427 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:35.279552 (+ 4125us) server_negotiation.cc:407] Connection header received
0812 07:12:35.279615 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:35.279619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:35.279683 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:35.279772 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:35.281727 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.282854 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.284850 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.285278 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.285391 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.289173 (+ 3782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:35.289181 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:35.289186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:35.289226 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:35.292764 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.293624 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.293629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.293632 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.293754 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:35.294274 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:35.294279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:35.294282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:35.294617 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:35.294751 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:35.295627 (+ 876us) server_negotiation.cc:299] Negotiation successful
0812 07:12:35.295837 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9500,"thread_start_us":9371,"threads_started":1}
I20260812 07:12:35.297237 14011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:35.266036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45651 (local address 127.21.181.66:46845)
0812 07:12:35.277484 (+ 11448us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:35.277530 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:35.279221 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:35.280104 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:35.280116 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:35.280737 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:35.281490 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:35.281518 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.283055 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.283061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:35.284681 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:35.284701 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:35.285503 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:35.285511 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:35.285730 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:35.286346 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:35.286368 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:35.288918 (+ 2550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:35.292933 (+ 4015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:35.292941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:35.292958 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:35.293483 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:35.293868 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:35.293873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:35.293876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:35.294139 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:35.294840 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:35.294848 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:35.295265 (+ 417us) client_negotiation.cc:769] Sending connection context
0812 07:12:35.295495 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 07:12:35.296013 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11140,"thread_start_us":13760,"threads_started":1}
I20260812 07:12:35.298761 14005 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45651
I20260812 07:12:35.299045 14005 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:35.299736 14005 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
I20260812 07:12:35.301718 13422 ts_manager.cc:194] Registered new tserver with Master: d439871a6fb34bf0bd950b06c039ec48 (127.21.181.66:37791)
I20260812 07:12:35.303862 13422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:46845
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:35.697657 13765 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:35.698112 13765 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
I20260812 07:12:35.699472 13422 ts_manager.cc:194] Registered new tserver with Master: 8511d54da6234bfba9b66c8488a8c857 (127.21.181.65:35545)
I20260812 07:12:35.700649 13422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:43341
W20260812 07:12:35.733762 14012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:35.734118 14012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:35.734320 14012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:35.744712 14012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:35.744867 14012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:35.745039 14012 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:12:35.758049 14012 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.21.181.84:35121/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:34025
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=33535
--tserver_master_addrs=127.21.181.126:45651
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:35.759311 14012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:35.760599 14012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:35.778656 14024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.778975 14030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:35.784372 14023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:35.789556 14012 server_base.cc:1034] running on GCE node
I20260812 07:12:35.790297 14012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:35.791529 14012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:35.792781 14012 hybrid_clock.cc:648] HybridClock initialized: now 1786518755792712 us; error 53 us; skew 500 ppm
I20260812 07:12:35.793169 14012 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:35.797295 14012 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:12:35.799309 14012 webserver.cc:460] Webserver started at http://127.21.181.67:33535/ using document root <none> and password file <none>
I20260812 07:12:35.800233 14012 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:35.800415 14012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:35.837641 14012 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.006s sys 0.005s
I20260812 07:12:35.841480 14049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.843343 14012 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:12:35.843560 14012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "53e13b407eaa4035bb7e126015c87567"
format_stamp: "Formatted at 2026-08-12 07:12:32 on dist-test-slave-lgr5"
server_key: "27a5decb074cf2328c64ab9961607e1c"
server_key_iv: "1bbc30ba6fda5e1bb7d5a7eaf1831b42"
server_key_version: "kuduclusterkey@0"
I20260812 07:12:35.845036 14012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:35.900113 14012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:35.905699 14012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:35.906111 14012 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 12 07:12:35 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for HTTP/127.21.181.84@KRBTEST.COM
I20260812 07:12:35.921200 14012 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:35.928781 14012 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:35.928920 14012 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:35.929042 14012 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:35.929121 14012 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:36.201308 14012 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:34025
I20260812 07:12:36.201400 14250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:34025 every 8 connection(s)
I20260812 07:12:36.203599 14012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:36.205091 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 14012
Aug 12 07:12:36 dist-test-slave-lgr5 krb5kdc[11763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518755, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:36.224715 14256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.206196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.21.181.126:45651)
0812 07:12:36.209976 (+ 3780us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.209983 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.210004 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:12:36.210088 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.210095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.210179 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.210310 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.212633 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.213898 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.215917 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.216431 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.216559 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.219269 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.219275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.219278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.219324 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.222066 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.222718 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.222723 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.222725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.222845 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.223187 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.223192 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.223194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.223528 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.223747 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.224174 (+ 427us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.224343 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3652,"thread_start_us":3493,"threads_started":1}
I20260812 07:12:36.229596 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.209894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:45383 (local address 127.21.181.126:45651)
0812 07:12:36.214687 (+ 4793us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.214693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.214710 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:12:36.214760 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.214764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.214843 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.215807 (+ 964us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.216750 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.217711 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.219190 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.219696 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.219801 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.222910 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.222915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.222918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.222947 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.226064 (+ 3117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.227443 (+ 1379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.227449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.227452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.227561 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.227982 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.227986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.227988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.228276 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.228408 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.229029 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.229230 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4675,"thread_start_us":4552,"threads_started":1}
I20260812 07:12:36.230449 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.209506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:45651 (local address 127.21.181.67:45383)
0812 07:12:36.211415 (+ 1909us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.211463 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.213973 (+ 2510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.215279 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.215293 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.215923 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:36.216575 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.216593 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.217858 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.217864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.219058 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.219070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.219909 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.219919 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.220186 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.220774 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:36.220794 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:36.222673 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:36.226810 (+ 4137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:36.226816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:36.226826 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:36.227319 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:36.227659 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:36.227665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:36.227669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:36.227859 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:36.228395 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:36.228403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:36.228702 (+ 299us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.228932 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.229410 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1622,"thread_start_us":477,"threads_started":1}
I20260812 07:12:36.231599 14251 heartbeater.cc:347] Connected to a master server at 127.21.181.126:45651
I20260812 07:12:36.231876 14251 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:36.232630 14251 heartbeater.cc:511] Master 127.21.181.126:45651 requested a full tablet report, sending...
I20260812 07:12:36.233960 13422 ts_manager.cc:194] Registered new tserver with Master: 53e13b407eaa4035bb7e126015c87567 (127.21.181.67:34025)
I20260812 07:12:36.234691 13422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:45383
I20260812 07:12:36.239023 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:36.253255 14262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.242021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56918 (local address 127.21.181.126:45651)
0812 07:12:36.242211 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.242217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.242443 (+ 226us) server_negotiation.cc:407] Connection header received
0812 07:12:36.242575 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.242578 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.242645 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.242724 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.244121 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.244779 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.246092 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.246465 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.246547 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.248830 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.248835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.248838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.248866 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.251074 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.251716 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.251719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.251721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.251825 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.252127 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.252130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.252132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.252368 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.252535 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.252785 (+ 250us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.252962 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:12:36.257870 13422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:36.265017 13422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:36.306094 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.278322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43948 (local address 127.21.181.65:35545)
0812 07:12:36.297581 (+ 19259us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.297592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.297623 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:12:36.297729 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.297737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.298016 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.298225 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.299066 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.300515 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.303469 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.304793 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.304935 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.305189 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.305544 (+ 355us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.305714 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19065,"thread_start_us":18907,"threads_started":1}
I20260812 07:12:36.306226 14291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.278129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:43948)
0812 07:12:36.290038 (+ 11909us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.290080 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.290272 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.298405 (+ 8133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.298412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.298441 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.298815 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.298829 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.300668 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.300674 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.303286 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.303304 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.305037 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.305044 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.305244 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.305318 (+ 74us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.305453 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.305777 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11614,"mutex_wait_us":33,"thread_start_us":12313,"threads_started":1}
I20260812 07:12:36.306793 14293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.280700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:37791 (local address 127.0.0.1:45488)
0812 07:12:36.297152 (+ 16452us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.297170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.297280 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.298947 (+ 1667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.298953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.298981 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.299294 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.299306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.301468 (+ 2162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.301474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.303874 (+ 2400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.303890 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.305921 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.305927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.306103 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.306135 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.306232 (+ 97us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.306347 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16346,"mutex_wait_us":71,"thread_start_us":15587,"threads_started":1}
I20260812 07:12:36.307163 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.278378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:34025 (local address 127.0.0.1:45316)
0812 07:12:36.296528 (+ 18150us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.296553 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.296676 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.299530 (+ 2854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.299535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.299565 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.299948 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.299963 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.301538 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.301544 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.303556 (+ 2012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.303571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.305384 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.305391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.305554 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.305585 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.305668 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.305789 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18029,"mutex_wait_us":120,"thread_start_us":18500,"threads_started":1}
I20260812 07:12:36.307191 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.278591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45316 (local address 127.21.181.67:34025)
0812 07:12:36.298993 (+ 20402us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.299003 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.299034 (+ 31us) server_negotiation.cc:407] Connection header received
0812 07:12:36.299128 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.299136 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.299341 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.299549 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.300179 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.301342 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.303730 (+ 2388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.304925 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.305067 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.305301 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.305962 (+ 661us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.306128 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20227,"thread_start_us":20040,"threads_started":1}
I20260812 07:12:36.307608 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.281012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.21.181.66:37791)
0812 07:12:36.297985 (+ 16973us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.297996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.298033 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:12:36.298345 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.298354 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.298589 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.298789 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.299477 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.301227 (+ 1750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.304040 (+ 2813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.305401 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.305541 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.305752 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.307099 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.307258 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16786,"thread_start_us":16599,"threads_started":1}
I20260812 07:12:36.308673 14005 heartbeater.cc:503] Master 127.21.181.126:45651 was elected leader, sending a full tablet report...
I20260812 07:12:36.310760 14134 tablet_service.cc:1467] Processing CreateTablet for tablet b05c2d8a45e7455baad5a2ee7f7469c3 (DEFAULT_TABLE table=test-table [id=90871fa0075a42d1af745603f94de28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:36.310942 13898 tablet_service.cc:1467] Processing CreateTablet for tablet b05c2d8a45e7455baad5a2ee7f7469c3 (DEFAULT_TABLE table=test-table [id=90871fa0075a42d1af745603f94de28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:36.310943 13654 tablet_service.cc:1467] Processing CreateTablet for tablet b05c2d8a45e7455baad5a2ee7f7469c3 (DEFAULT_TABLE table=test-table [id=90871fa0075a42d1af745603f94de28b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:36.311998 14134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05c2d8a45e7455baad5a2ee7f7469c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.312366 13654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05c2d8a45e7455baad5a2ee7f7469c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.312361 13898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b05c2d8a45e7455baad5a2ee7f7469c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:36.323966 14299 tablet_bootstrap.cc:492] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Bootstrap starting.
I20260812 07:12:36.326924 14299 tablet_bootstrap.cc:654] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.327809 14300 tablet_bootstrap.cc:492] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Bootstrap starting.
I20260812 07:12:36.328238 14299 log.cc:824] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.331461 14300 tablet_bootstrap.cc:654] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.331759 14301 tablet_bootstrap.cc:492] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Bootstrap starting.
I20260812 07:12:36.332129 14299 tablet_bootstrap.cc:492] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: No bootstrap required, opened a new log
I20260812 07:12:36.332396 14299 ts_tablet_manager.cc:1397] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260812 07:12:36.332749 14300 log.cc:824] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.334906 14301 tablet_bootstrap.cc:654] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:36.335844 14300 tablet_bootstrap.cc:492] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: No bootstrap required, opened a new log
I20260812 07:12:36.335904 14301 log.cc:824] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:36.336122 14300 ts_tablet_manager.cc:1397] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:12:36.338464 14301 tablet_bootstrap.cc:492] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: No bootstrap required, opened a new log
I20260812 07:12:36.338696 14301 ts_tablet_manager.cc:1397] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260812 07:12:36.342445 14299 raft_consensus.cc:348] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.342892 14299 raft_consensus.cc:374] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.343001 14299 raft_consensus.cc:729] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d439871a6fb34bf0bd950b06c039ec48, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.343597 14299 consensus_queue.cc:260] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.346467 14299 ts_tablet_manager.cc:1428] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260812 07:12:36.346192 14300 raft_consensus.cc:348] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.346606 14300 raft_consensus.cc:374] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.346699 14300 raft_consensus.cc:729] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53e13b407eaa4035bb7e126015c87567, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.347191 14300 consensus_queue.cc:260] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.347167 14301 raft_consensus.cc:348] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.347604 14301 raft_consensus.cc:374] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:36.347716 14301 raft_consensus.cc:729] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8511d54da6234bfba9b66c8488a8c857, State: Initialized, Role: FOLLOWER
I20260812 07:12:36.348317 14251 heartbeater.cc:503] Master 127.21.181.126:45651 was elected leader, sending a full tablet report...
I20260812 07:12:36.348548 14301 consensus_queue.cc:260] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.348814 14300 ts_tablet_manager.cc:1428] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260812 07:12:36.349562 13765 heartbeater.cc:503] Master 127.21.181.126:45651 was elected leader, sending a full tablet report...
I20260812 07:12:36.350117 14301 ts_tablet_manager.cc:1428] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
W20260812 07:12:36.404580 13766 tablet.cc:2369] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:36.415000 14310 raft_consensus.cc:482] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:36.415266 14310 raft_consensus.cc:504] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.417030 14310 leader_election.cc:288] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8511d54da6234bfba9b66c8488a8c857 (127.21.181.65:35545), d439871a6fb34bf0bd950b06c039ec48 (127.21.181.66:37791)
I20260812 07:12:36.422444 14260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.417189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.21.181.67:48799)
0812 07:12:36.417462 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.417493 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.417614 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.418001 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.418007 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.418042 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.418250 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.418260 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.419273 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.419277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.420624 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.420639 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.421760 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.421766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.421946 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.421971 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.422065 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.422187 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260812 07:12:36.422449 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.417392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:48799 (local address 127.21.181.65:35545)
0812 07:12:36.417597 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.417603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.417623 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:36.417763 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.417768 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.417845 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.417935 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.418401 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.419131 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.420783 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.421619 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.421765 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.421839 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.422105 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.422225 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260812 07:12:36.423264 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.417616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:51593 (local address 127.21.181.66:37791)
0812 07:12:36.417818 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.417825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.418303 (+ 478us) server_negotiation.cc:407] Connection header received
0812 07:12:36.418453 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.418456 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.418515 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.418586 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:36.419110 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.419818 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.421490 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.422372 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.422517 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.422586 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.422904 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.423035 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:12:36.423295 14313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.417432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:37791 (local address 127.21.181.67:51593)
0812 07:12:36.418196 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.418210 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:36.418326 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:36.418619 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:36.418624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:36.418648 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:36.418962 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.418971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.419967 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.419971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.421336 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:36.421350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.422571 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.422578 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:36.422756 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.422780 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:12:36.422858 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:12:36.422974 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":438,"threads_started":1}
I20260812 07:12:36.423429 13689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05c2d8a45e7455baad5a2ee7f7469c3" candidate_uuid: "53e13b407eaa4035bb7e126015c87567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8511d54da6234bfba9b66c8488a8c857" is_pre_election: true
I20260812 07:12:36.423987 13689 raft_consensus.cc:2454] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53e13b407eaa4035bb7e126015c87567 in term 0.
I20260812 07:12:36.424363 13928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05c2d8a45e7455baad5a2ee7f7469c3" candidate_uuid: "53e13b407eaa4035bb7e126015c87567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d439871a6fb34bf0bd950b06c039ec48" is_pre_election: true
I20260812 07:12:36.424628 14052 leader_election.cc:302] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53e13b407eaa4035bb7e126015c87567, 8511d54da6234bfba9b66c8488a8c857; no voters:
I20260812 07:12:36.424818 13928 raft_consensus.cc:2454] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53e13b407eaa4035bb7e126015c87567 in term 0.
I20260812 07:12:36.425024 14310 raft_consensus.cc:2781] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:36.425174 14310 raft_consensus.cc:482] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:36.425292 14310 raft_consensus.cc:3037] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.427525 14310 raft_consensus.cc:504] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.428020 14310 leader_election.cc:288] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [CANDIDATE]: Term 1 election: Requested vote from peers 8511d54da6234bfba9b66c8488a8c857 (127.21.181.65:35545), d439871a6fb34bf0bd950b06c039ec48 (127.21.181.66:37791)
I20260812 07:12:36.428439 13689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05c2d8a45e7455baad5a2ee7f7469c3" candidate_uuid: "53e13b407eaa4035bb7e126015c87567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8511d54da6234bfba9b66c8488a8c857"
I20260812 07:12:36.428484 13928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b05c2d8a45e7455baad5a2ee7f7469c3" candidate_uuid: "53e13b407eaa4035bb7e126015c87567" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d439871a6fb34bf0bd950b06c039ec48"
I20260812 07:12:36.428622 13689 raft_consensus.cc:3037] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.428642 13928 raft_consensus.cc:3037] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:36.430692 13689 raft_consensus.cc:2454] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53e13b407eaa4035bb7e126015c87567 in term 1.
I20260812 07:12:36.430697 13928 raft_consensus.cc:2454] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53e13b407eaa4035bb7e126015c87567 in term 1.
I20260812 07:12:36.431056 14051 leader_election.cc:302] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53e13b407eaa4035bb7e126015c87567, d439871a6fb34bf0bd950b06c039ec48; no voters:
I20260812 07:12:36.431357 14310 raft_consensus.cc:2781] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:36.431794 14310 raft_consensus.cc:686] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 1 LEADER]: Becoming Leader. State: Replica: 53e13b407eaa4035bb7e126015c87567, State: Running, Role: LEADER
I20260812 07:12:36.432256 14310 consensus_queue.cc:237] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } }
I20260812 07:12:36.436564 13419 catalog_manager.cc:5495] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 reported cstate change: term changed from 0 to 1, leader changed from <none> to 53e13b407eaa4035bb7e126015c87567 (127.21.181.67). New cstate: current_term: 1 leader_uuid: "53e13b407eaa4035bb7e126015c87567" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "53e13b407eaa4035bb7e126015c87567" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 34025 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 } health_report { overall_health: UNKNOWN } } }
I20260812 07:12:36.452687 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.447344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.21.181.67:34025)
0812 07:12:36.447540 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.447546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.447618 (+ 72us) server_negotiation.cc:407] Connection header received
0812 07:12:36.447819 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.447823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.447892 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.448014 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:36.448443 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.449175 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.450644 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.451034 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.451165 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.451626 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:36.452138 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:36.452266 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.452371 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.452448 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260812 07:12:36.456027 14253 tablet.cc:2369] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:36.460264 13689 raft_consensus.cc:1261] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 1 FOLLOWER]: Refusing update from remote peer 53e13b407eaa4035bb7e126015c87567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:36.460264 13928 raft_consensus.cc:1261] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 1 FOLLOWER]: Refusing update from remote peer 53e13b407eaa4035bb7e126015c87567: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:36.461293 14314 consensus_queue.cc:1035] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d439871a6fb34bf0bd950b06c039ec48" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 37791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:36.461658 14310 consensus_queue.cc:1035] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8511d54da6234bfba9b66c8488a8c857" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:36.472425 14317 mvcc.cc:205] Tried to move back new op lower bound from 7317580826451677184 to 7317580826350276608. Current Snapshot: MvccSnapshot[applied={T|T < 7317580826451677184}]
I20260812 07:12:36.486802 13419 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:36.487100 13419 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:36.490602 13419 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53b267ad1d774f53829f0b152d739cb6: Sending DeleteTablet for 3 replicas of tablet b05c2d8a45e7455baad5a2ee7f7469c3
I20260812 07:12:36.491739 13654 tablet_service.cc:1514] Processing DeleteTablet for tablet b05c2d8a45e7455baad5a2ee7f7469c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:36 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:43948
I20260812 07:12:36.491750 14134 tablet_service.cc:1514] Processing DeleteTablet for tablet b05c2d8a45e7455baad5a2ee7f7469c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:36 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:45316
I20260812 07:12:36.491855 13898 tablet_service.cc:1514] Processing DeleteTablet for tablet b05c2d8a45e7455baad5a2ee7f7469c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:36 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:45488
I20260812 07:12:36.501091 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 13489
I20260812 07:12:36.507328 14332 tablet_replica.cc:330] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: stopping tablet replica
I20260812 07:12:36.507859 14332 raft_consensus.cc:2229] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:36.508486 14332 raft_consensus.cc:2258] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:36.509315 14333 tablet_replica.cc:330] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: stopping tablet replica
I20260812 07:12:36.510742 14333 raft_consensus.cc:2229] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:36.511436 14333 raft_consensus.cc:2258] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:36.515092 14332 ts_tablet_manager.cc:1905] T b05c2d8a45e7455baad5a2ee7f7469c3 P 8511d54da6234bfba9b66c8488a8c857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:36.517803 14333 ts_tablet_manager.cc:1905] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:36.520455 14292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.502350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.21.181.65:35545)
0812 07:12:36.502522 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.502528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.502676 (+ 148us) server_negotiation.cc:407] Connection header received
0812 07:12:36.502860 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.502865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.502943 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.503031 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.505568 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.506799 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.509388 (+ 2589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.509983 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.510130 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.513605 (+ 3475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.513616 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.513631 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.513667 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.517428 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.518210 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.518219 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.518222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.518418 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.518791 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.518796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.518799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.519227 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.519556 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.519779 (+ 223us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.519988 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:12:36.521363 13754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:36.522512 14336 tablet_replica.cc:330] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: stopping tablet replica
I20260812 07:12:36.522961 14336 raft_consensus.cc:2229] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:36.523471 14336 raft_consensus.cc:2258] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:36.525578 14336 ts_tablet_manager.cc:1905] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:36.532555 14333 ts_tablet_manager.cc:1918] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:36.532611 14336 ts_tablet_manager.cc:1918] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:36.532773 14336 log.cc:1196] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/b05c2d8a45e7455baad5a2ee7f7469c3
I20260812 07:12:36.533305 14336 ts_tablet_manager.cc:1939] T b05c2d8a45e7455baad5a2ee7f7469c3 P d439871a6fb34bf0bd950b06c039ec48: Deleting consensus metadata
I20260812 07:12:36.533690 14333 log.cc:1196] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/b05c2d8a45e7455baad5a2ee7f7469c3
I20260812 07:12:36.534324 14333 ts_tablet_manager.cc:1939] T b05c2d8a45e7455baad5a2ee7f7469c3 P 53e13b407eaa4035bb7e126015c87567: Deleting consensus metadata
I20260812 07:12:36.535254 13399 catalog_manager.cc:4841] TS d439871a6fb34bf0bd950b06c039ec48 (127.21.181.66:37791): tablet b05c2d8a45e7455baad5a2ee7f7469c3 (table test-table [id=90871fa0075a42d1af745603f94de28b]) successfully deleted
I20260812 07:12:36.536095 13402 catalog_manager.cc:4841] TS 53e13b407eaa4035bb7e126015c87567 (127.21.181.67:34025): tablet b05c2d8a45e7455baad5a2ee7f7469c3 (table test-table [id=90871fa0075a42d1af745603f94de28b]) successfully deleted
I20260812 07:12:36.621349 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 13489
W20260812 07:12:36.643512 13401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.181.65:35545 (error 108)
I20260812 07:12:36.643775 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 13771
I20260812 07:12:36.645592 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.645085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:43972)
0812 07:12:36.645276 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.645402 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260812 07:12:36.645869 13401 catalog_manager.cc:4584] TS 8511d54da6234bfba9b66c8488a8c857 (127.21.181.65:35545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b05c2d8a45e7455baad5a2ee7f7469c3: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
I20260812 07:12:36.662056 14298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.644658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.21.181.66:37791)
0812 07:12:36.644855 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.644859 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.644924 (+ 65us) server_negotiation.cc:407] Connection header received
0812 07:12:36.645108 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.645113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.645214 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.645314 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.646732 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.647468 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.648888 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.649284 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.649391 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.655807 (+ 6416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.655821 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.655835 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.655868 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.658608 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.659270 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.659277 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.659279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.659407 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.659761 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.659765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.659768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.660280 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.660605 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.661497 (+ 892us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.661712 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:12:36.662747 13997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:36.700553 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.699771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:43986)
0812 07:12:36.700068 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.700281 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260812 07:12:36.701795 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.701407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:43990)
0812 07:12:36.701561 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.701665 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:12:36.751442 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 13771
I20260812 07:12:36.762532 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.762010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:43998)
0812 07:12:36.762244 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.762352 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260812 07:12:36.763734 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.763304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:44008)
0812 07:12:36.763486 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.763584 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:12:36.775388 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 14012
I20260812 07:12:36.789108 14295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.776118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.21.181.67:34025)
0812 07:12:36.776299 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:36.776305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:36.776387 (+ 82us) server_negotiation.cc:407] Connection header received
0812 07:12:36.776614 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:36.776619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:36.776685 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:36.776789 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:36.777985 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.778803 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.780260 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:36.780691 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:36.780811 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:36.783192 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:36.783203 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:36.783218 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:36.783253 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:36.786276 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.786913 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.786920 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.786922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.787063 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:36.787432 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:36.787437 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:36.787440 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:36.787834 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:36.788166 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:36.788567 (+ 401us) server_negotiation.cc:299] Negotiation successful
0812 07:12:36.788770 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:12:36.789822 14232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:36.844460 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.843978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:44014)
0812 07:12:36.844206 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.844303 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:12:36.845564 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:36.845173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:35545 (local address 127.0.0.1:44016)
0812 07:12:36.845375 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:36.845448 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260812 07:12:36.908453 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 14012
I20260812 07:12:36.938668 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 13375
I20260812 07:12:36.939812 13476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:37.048411 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 13375
I20260812 07:12:37.077278 22229 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260812 07:12:37.421201 22229 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260812 07:12:37.421409 22229 mini_ranger.cc:66] Stopping Ranger...
I20260812 07:12:37.764937 22229 mini_ranger.cc:68] Stopped Ranger
2026-08-12 07:12:37.765 UTC [11799] LOG: received smart shutdown request
2026-08-12 07:12:37.767 UTC [11799] LOG: background worker "logical replication launcher" (PID 11808) exited with exit code 1
2026-08-12 07:12:37.767 UTC [11803] LOG: shutting down
2026-08-12 07:12:37.890 UTC [11799] LOG: database system is shut down
2026-08-12T07:12:37Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50804 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:38 dist-test-slave-lgr5 krb5kdc[14358](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:38 dist-test-slave-lgr5 krb5kdc[14358](info): set up 2 sockets
Aug 12 07:12:38 dist-test-slave-lgr5 krb5kdc[14358](info): commencing operation
krb5kdc: starting...
W20260812 07:12:39.280836 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:12:39 dist-test-slave-lgr5 krb5kdc[14358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:12:39.506883 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44707
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:44707,127.21.181.125:40955,127.21.181.124:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:39.825703 14378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:39.825961 14378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:39.826097 14378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:39.832947 14378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:39.833050 14378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:39.833127 14378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:39.833202 14378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:39.833261 14378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:39.842509 14378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:44707,127.21.181.125:40955,127.21.181.124:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:39.843690 14378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:39.845098 14378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:39.855752 14389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:39.855867 14384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:39.856215 14400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:39.856570 14378 server_base.cc:1034] running on GCE node
I20260812 07:12:39.857199 14378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:39.858387 14378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:39.859584 14378 hybrid_clock.cc:648] HybridClock initialized: now 1786518759859536 us; error 37 us; skew 500 ppm
I20260812 07:12:39.859891 14378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:39 dist-test-slave-lgr5 krb5kdc[14358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:39.866621 14378 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:39.867926 14378 webserver.cc:460] Webserver started at http://127.21.181.126:35341/ using document root <none> and password file <none>
I20260812 07:12:39.868593 14378 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:39.868707 14378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:39.868981 14378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:39.871755 14378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "5e9127771a214304a130b1d943d9f8cb"
format_stamp: "Formatted at 2026-08-12 07:12:39 on dist-test-slave-lgr5"
server_key: "9a01fd0736c5352ed99c162fecf45552"
server_key_iv: "5d80c68c27eaf34663f849d2e366275c"
server_key_version: "encryptionkey@0"
I20260812 07:12:39.872548 14378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "5e9127771a214304a130b1d943d9f8cb"
format_stamp: "Formatted at 2026-08-12 07:12:39 on dist-test-slave-lgr5"
server_key: "9a01fd0736c5352ed99c162fecf45552"
server_key_iv: "5d80c68c27eaf34663f849d2e366275c"
server_key_version: "encryptionkey@0"
I20260812 07:12:39.877282 14378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:39.880383 14408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:39.881557 14378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:12:39.881701 14378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "5e9127771a214304a130b1d943d9f8cb"
format_stamp: "Formatted at 2026-08-12 07:12:39 on dist-test-slave-lgr5"
server_key: "9a01fd0736c5352ed99c162fecf45552"
server_key_iv: "5d80c68c27eaf34663f849d2e366275c"
server_key_version: "encryptionkey@0"
I20260812 07:12:39.881861 14378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:39.894924 14378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:39.898437 14378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:39.898684 14378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:39.914424 14378 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:44707
I20260812 07:12:39.914460 14500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:44707 every 8 connection(s)
I20260812 07:12:39.915664 14378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:39.919061 14501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:39.925009 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 14378
I20260812 07:12:39.925218 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:12:39.925079 14501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:12:39.925851 22229 external_mini_cluster.cc:1428] Setting key b02bd72d1cef1f04f3b63c05c6de7f78
Aug 12 07:12:39 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:39.946645 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:39.927965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44707 (local address 127.0.0.1:45874)
0812 07:12:39.928812 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:39.928865 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:39.931096 (+ 2231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:39.931875 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:39.931881 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:39.932592 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:39.933010 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:39.933033 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.934503 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.934507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:39.935945 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:39.935959 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.936854 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.936861 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:39.937010 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:39.937651 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:39.937672 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:39.939684 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:39.943237 (+ 3553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:39.943247 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:39.943254 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:39.943634 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:39.944044 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:39.944047 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:39.944049 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:39.944227 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:39.945010 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:39.945015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:39.945199 (+ 184us) client_negotiation.cc:769] Sending connection context
0812 07:12:39.945375 (+ 176us) client_negotiation.cc:240] Negotiation successful
0812 07:12:39.945706 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":351,"threads_started":1}
I20260812 07:12:39.946669 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:39.927965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45874 (local address 127.21.181.126:44707)
0812 07:12:39.929646 (+ 1681us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:39.929652 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:39.929699 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:12:39.931431 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:39.931436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:39.931647 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:39.931794 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:39.933210 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.934340 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.936069 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:39.936627 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:39.936745 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:39.939921 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:39.939935 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:39.939951 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:39.939989 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:39.943120 (+ 3131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:39.943762 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:39.943769 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:39.943773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:39.943939 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:39.944329 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:39.944334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:39.944337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:39.944905 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:39.945095 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:39.945561 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 07:12:39.945884 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":1501,"spinlock_wait_cycles":15744,"thread_start_us":1248,"threads_started":1}
I20260812 07:12:39.949378 14501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:12:39.950239 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:39.949913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33402)
0812 07:12:39.950055 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:39.950150 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:12:39.950477 14414 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
I20260812 07:12:39.951905 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:39.951604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33408)
0812 07:12:39.951730 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:39.951790 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:12:39.952123 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:40955: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.181.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab.
Entry for principal kudu/127.21.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab.
I20260812 07:12:39.984009 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:40955
--webserver_interface=127.21.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab
--principal=kudu/127.21.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:44707,127.21.181.125:40955,127.21.181.124:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:40.001420 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } attempt: 1
I20260812 07:12:40.002617 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.002153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33424)
0812 07:12:40.002353 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.002462 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:12:40.003599 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.003251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33432)
0812 07:12:40.003392 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.003462 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:12:40.003768 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:40955: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
I20260812 07:12:40.071969 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } attempt: 2
I20260812 07:12:40.073036 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.072592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33434)
0812 07:12:40.072818 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.072902 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:12:40.074066 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.073717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33438)
0812 07:12:40.073856 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.073931 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:12:40.074255 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:40955: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
I20260812 07:12:40.165503 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } attempt: 3
I20260812 07:12:40.166723 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.166284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33442)
0812 07:12:40.166491 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.166587 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:12:40.167804 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.167418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33456)
0812 07:12:40.167576 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.167664 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:12:40.168005 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:40955: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
W20260812 07:12:40.306815 14515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:40.307089 14515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:40.307224 14515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 07:12:40.311241 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } attempt: 4
I20260812 07:12:40.312366 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.311836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33472)
0812 07:12:40.312140 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.312229 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260812 07:12:40.313375 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.313029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33480)
0812 07:12:40.313169 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.313257 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:12:40.313541 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:40955: Network error: Client connection negotiation failed: client connection to 127.21.181.125:40955: connect: Connection refused (error 111)
W20260812 07:12:40.314373 14515 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:40.314477 14515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:40.314553 14515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:40.314630 14515 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:40.314687 14515 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:40.324050 14515 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:44707,127.21.181.125:40955,127.21.181.124:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab
--principal=kudu/127.21.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:40955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:40.325249 14515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:40.326472 14515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:40.337381 14526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:40.337313 14527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:40.337353 14532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:40.338256 14515 server_base.cc:1034] running on GCE node
I20260812 07:12:40.338935 14515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:40.340219 14515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:40.341425 14515 hybrid_clock.cc:648] HybridClock initialized: now 1786518760341374 us; error 41 us; skew 500 ppm
I20260812 07:12:40.341734 14515 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:40.344668 14515 init.cc:377] Logged in from keytab as kudu/127.21.181.125@KRBTEST.COM (short username kudu)
I20260812 07:12:40.346027 14515 webserver.cc:460] Webserver started at http://127.21.181.125:34085/ using document root <none> and password file <none>
I20260812 07:12:40.346645 14515 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:40.346756 14515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:40.347029 14515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:40.349849 14515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data/instance:
uuid: "d52cc5090a9a4f4495d959072930fadd"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-lgr5"
server_key: "7751ccb7501f6ee9c85d42f73d6d7895"
server_key_iv: "876e75a90f93f2246a12c0d5f2fc82b7"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.350632 14515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal/instance:
uuid: "d52cc5090a9a4f4495d959072930fadd"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-lgr5"
server_key: "7751ccb7501f6ee9c85d42f73d6d7895"
server_key_iv: "876e75a90f93f2246a12c0d5f2fc82b7"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.355422 14515 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260812 07:12:40.358703 14550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:40.359871 14515 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:12:40.360028 14515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal
uuid: "d52cc5090a9a4f4495d959072930fadd"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-lgr5"
server_key: "7751ccb7501f6ee9c85d42f73d6d7895"
server_key_iv: "876e75a90f93f2246a12c0d5f2fc82b7"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.360229 14515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:40.374174 14515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:40.377627 14515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:40.377861 14515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:40.394379 14515 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.125:40955
I20260812 07:12:40.394412 14626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.125:40955 every 8 connection(s)
I20260812 07:12:40.395576 14515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
I20260812 07:12:40.398898 14634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:40.405356 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 14515
I20260812 07:12:40.405539 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-1/wal/instance
I20260812 07:12:40.404896 14634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:12:40.406100 22229 external_mini_cluster.cc:1428] Setting key 5d7be69d7a3544c3e27768dd174752bf
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:40.425766 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.407825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45876 (local address 127.21.181.126:44707)
0812 07:12:40.408023 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.408029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.409129 (+ 1100us) server_negotiation.cc:407] Connection header received
0812 07:12:40.411328 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.411333 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.411396 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.411495 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.413286 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.413969 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.415975 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.416421 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.416529 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.420176 (+ 3647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.420181 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.420184 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.420217 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.422657 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.423465 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.423469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.423471 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.423590 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.424040 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.424043 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.424045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.424336 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.424445 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.425227 (+ 782us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.425445 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:12:40.426826 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.408225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44707 (local address 127.0.0.1:45876)
0812 07:12:40.408950 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.409003 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:40.411072 (+ 2069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:40.411723 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:40.411732 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:40.412412 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:40.413127 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.413153 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.414131 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.414136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.415831 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.415846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.416638 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.416645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.416885 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.417538 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:40.417564 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:40.419915 (+ 2351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:40.422793 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.422802 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.422816 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.423348 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.423691 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.423695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.423698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.423940 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.424463 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:40.424471 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:40.424832 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 07:12:40.425147 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 07:12:40.425575 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":299,"threads_started":1}
I20260812 07:12:40.428539 14634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.181.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.124@KRBTEST.COM" created.
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:12:40.443044 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.429115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33496)
0812 07:12:40.429246 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.429260 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:40.429335 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:40.430250 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:40.430255 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:40.430678 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:40.430854 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.430865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.432153 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.432159 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.433449 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.433465 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.434227 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.434233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.434423 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.434850 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:40.434865 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:40.437489 (+ 2624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:40.440579 (+ 3090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.440584 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.440587 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.441052 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.441381 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.441385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.441388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.441565 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.442140 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:40.442145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:40.442272 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 07:12:40.442412 (+ 140us) client_negotiation.cc:240] Negotiation successful
0812 07:12:40.442695 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:12:40.443130 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.429346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.21.181.125:40955)
0812 07:12:40.429840 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.429845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.429868 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:12:40.429907 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.429912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.430072 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.430225 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.431003 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.431995 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.433577 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.434076 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.434195 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.437677 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.437688 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.437699 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.437725 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.440436 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.441164 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.441171 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.441173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.441286 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.441665 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.441669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.441672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.442015 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.442178 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.442537 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.442802 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":365,"thread_start_us":246,"threads_started":1}
I20260812 07:12:40.445536 14634 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:12:40.446539 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.446142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41196)
0812 07:12:40.446300 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.446393 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260812 07:12:40.446815 14551 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.448247 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.447930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41200)
0812 07:12:40.448057 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.448160 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:12:40.448454 14634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab.
Entry for principal kudu/127.21.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab.
I20260812 07:12:40.465034 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:44521
--webserver_interface=127.21.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab
--principal=kudu/127.21.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:41577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:44707,127.21.181.125:40955,127.21.181.124:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:40.497726 14634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 1
I20260812 07:12:40.498813 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.498394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41202)
0812 07:12:40.498577 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.498672 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 07:12:40.499732 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.499416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41212)
0812 07:12:40.499567 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.499632 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260812 07:12:40.499943 14634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.568152 14634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 2
I20260812 07:12:40.569186 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.568778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41220)
0812 07:12:40.568952 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.569038 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:12:40.570130 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.569798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41234)
0812 07:12:40.569944 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.570015 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:12:40.570363 14634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.612766 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } attempt: 5
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:12:40.625550 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.613484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33504)
0812 07:12:40.613709 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.613731 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:40.613864 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:40.614208 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:40.614213 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:40.614608 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:40.614792 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.614801 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.615827 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.615831 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.616752 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.616765 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.617471 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.617478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.617649 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.618217 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:40.618235 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:40.620475 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:40.623245 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.623250 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.623255 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.623601 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.623942 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.623948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.623951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.624119 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.624623 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:40.624627 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:40.624777 (+ 150us) client_negotiation.cc:769] Sending connection context
0812 07:12:40.624892 (+ 115us) client_negotiation.cc:240] Negotiation successful
0812 07:12:40.625186 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:12:40.625561 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.613573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.21.181.125:40955)
0812 07:12:40.613775 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.613781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.613803 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:12:40.613998 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.614002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.614067 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.614151 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.614965 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.615692 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.616899 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.617313 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.617426 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.620652 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.620658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.620661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.620695 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.623109 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.623712 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.623716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.623719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.623840 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.624211 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.624216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.624220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.624509 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.624644 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.625037 (+ 393us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.625257 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260812 07:12:40.626470 14501 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:12:40.627521 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.627071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41250)
0812 07:12:40.627249 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.627344 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260812 07:12:40.628562 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.628183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41256)
0812 07:12:40.628322 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.628420 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 07:12:40.628775 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.661602 14634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 3
I20260812 07:12:40.662593 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.662165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41270)
0812 07:12:40.662347 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.662441 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:12:40.663658 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.663243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41272)
0812 07:12:40.663460 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.663541 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260812 07:12:40.663872 14634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.679983 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 1
I20260812 07:12:40.680953 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.680491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41280)
0812 07:12:40.680678 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.680778 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:12:40.681958 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.681596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41284)
0812 07:12:40.681732 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.681824 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:12:40.682191 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.750388 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 2
I20260812 07:12:40.751578 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.750972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41294)
0812 07:12:40.751242 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.751359 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260812 07:12:40.752642 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.752250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41302)
0812 07:12:40.752402 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.752506 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:12:40.752919 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
W20260812 07:12:40.782344 14645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:40.782572 14645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:40.782701 14645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:40.789646 14645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:40.789742 14645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:40.789814 14645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:40.789885 14645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:40.789943 14645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:40.799201 14645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:41577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:44707,127.21.181.125:40955,127.21.181.124:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab
--principal=kudu/127.21.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:40.800401 14645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:40.801605 14645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:12:40.807080 14634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 4
I20260812 07:12:40.808229 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.807748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41318)
0812 07:12:40.807948 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.808042 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:12:40.809231 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.808919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41332)
0812 07:12:40.809059 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.809129 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:12:40.809413 14634 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
W20260812 07:12:40.812811 14655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:40.812776 14661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:40.812879 14654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:40.813570 14645 server_base.cc:1034] running on GCE node
I20260812 07:12:40.814288 14645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:40.815575 14645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:40.816767 14645 hybrid_clock.cc:648] HybridClock initialized: now 1786518760816718 us; error 38 us; skew 500 ppm
I20260812 07:12:40.817092 14645 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:40.819955 14645 init.cc:377] Logged in from keytab as kudu/127.21.181.124@KRBTEST.COM (short username kudu)
I20260812 07:12:40.821288 14645 webserver.cc:460] Webserver started at http://127.21.181.124:39697/ using document root <none> and password file <none>
I20260812 07:12:40.821924 14645 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:40.822037 14645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:40.822320 14645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:40.825156 14645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data/instance:
uuid: "3e35a5c1f0554655a488d2a00544e22f"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-lgr5"
server_key: "547c412fcb7a265995d0d6fc51f14ac7"
server_key_iv: "c4c58180b6cf701f147c0a2ed9aa91b1"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.825929 14645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal/instance:
uuid: "3e35a5c1f0554655a488d2a00544e22f"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-lgr5"
server_key: "547c412fcb7a265995d0d6fc51f14ac7"
server_key_iv: "c4c58180b6cf701f147c0a2ed9aa91b1"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.830646 14645 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:12:40.833792 14672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:40.834949 14645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:12:40.835109 14645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal
uuid: "3e35a5c1f0554655a488d2a00544e22f"
format_stamp: "Formatted at 2026-08-12 07:12:40 on dist-test-slave-lgr5"
server_key: "547c412fcb7a265995d0d6fc51f14ac7"
server_key_iv: "c4c58180b6cf701f147c0a2ed9aa91b1"
server_key_version: "encryptionkey@0"
I20260812 07:12:40.835278 14645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:40.858770 14645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:40.859110 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 3
I20260812 07:12:40.860122 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.859654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41338)
0812 07:12:40.859832 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.859922 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:12:40.861114 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.860737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41352)
0812 07:12:40.860920 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.861009 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260812 07:12:40.861312 14501 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44521: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44521: connect: Connection refused (error 111)
I20260812 07:12:40.863118 14645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:40.863394 14645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:40.879552 14645 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.124:44521
I20260812 07:12:40.879581 14761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.124:44521 every 8 connection(s)
I20260812 07:12:40.880915 14645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
I20260812 07:12:40.884336 14762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:40.884699 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 14645
I20260812 07:12:40.884909 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786518562800324-22229-0/minicluster-data/master-2/wal/instance
I20260812 07:12:40.885386 22229 external_mini_cluster.cc:1428] Setting key 7e566b05e1500c73bffafcd67bdb60ed
I20260812 07:12:40.893829 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:40.908776 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.887126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41358 (local address 127.21.181.124:44521)
0812 07:12:40.887894 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.887909 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.887960 (+ 51us) server_negotiation.cc:407] Connection header received
0812 07:12:40.890585 (+ 2625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.890612 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.891075 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.891522 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.893188 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.895152 (+ 1964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.896704 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.897698 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.897878 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.902229 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.902237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.902247 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.902285 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.905044 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.905691 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.905698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.905703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.905847 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.906234 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.906238 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.906240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.906702 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.906975 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.907404 (+ 429us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.907833 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":347,"threads_started":1}
I20260812 07:12:40.909968 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.896513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.21.181.126:44707)
0812 07:12:40.896665 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.896670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.897356 (+ 686us) server_negotiation.cc:407] Connection header received
0812 07:12:40.897527 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.897531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.897592 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.897661 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.898793 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.899810 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.901113 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.901771 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.901905 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.905102 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.905106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.905110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.905140 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.907510 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.908186 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.908191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.908195 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.908328 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.908678 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.908681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.908684 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.908928 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.909068 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.909577 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.909735 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:12:40.910147 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.896516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44707 (local address 127.0.0.1:45886)
0812 07:12:40.897206 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.897260 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:40.897409 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:40.897744 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:40.897751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:40.898395 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:40.898637 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.898645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.899938 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.899942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.900978 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.900990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.901980 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.901985 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.902114 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.902680 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:40.902698 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:40.904924 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:40.907632 (+ 2708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.907644 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.907651 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.908060 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.908435 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.908440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.908444 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.908570 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.909061 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:40.909068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:40.909275 (+ 207us) client_negotiation.cc:769] Sending connection context
0812 07:12:40.909479 (+ 204us) client_negotiation.cc:240] Negotiation successful
0812 07:12:40.909759 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":319,"threads_started":1}
I20260812 07:12:40.911712 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:40.925554 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.912343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:40955 (local address 127.0.0.1:33510)
0812 07:12:40.912569 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.912585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:40.912666 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:40.913011 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:40.913014 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:40.913298 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:40.913421 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.913428 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.914526 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.914531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.915606 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.915622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.916591 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.916596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.916798 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.917716 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:40.917732 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:40.920237 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:40.923359 (+ 3122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.923363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.923365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.923696 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.924063 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.924068 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.924070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.924234 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.924715 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:40.924720 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:40.924861 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:12:40.924974 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 07:12:40.925227 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260812 07:12:40.925665 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.912550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.21.181.125:40955)
0812 07:12:40.912736 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.912742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.912761 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:12:40.912821 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.912827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.912901 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.913006 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.913562 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.914394 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.915765 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.916441 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.916553 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.920440 (+ 3887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.920447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.920451 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.920495 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.923235 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.923812 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.923817 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.923821 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.923955 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.924339 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.924345 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.924348 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.924602 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.924731 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.925095 (+ 364us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.925302 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:12:40.926622 14762 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:12:40.927043 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.911889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45896 (local address 127.21.181.126:44707)
0812 07:12:40.912163 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.912167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.912181 (+ 14us) server_negotiation.cc:407] Connection header received
0812 07:12:40.912308 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.912312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.912378 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.912463 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.913884 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.914977 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.916465 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.917081 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.917184 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.921544 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.921550 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.921554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.921589 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.924539 (+ 2950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.925184 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.925189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.925191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.925308 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.925746 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.925751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.925755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.926034 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.926287 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.926574 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.926774 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
Aug 12 07:12:40 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:12:40.939189 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.927361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41364)
0812 07:12:40.927504 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:40.927521 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:40.927612 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:40.927927 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:40.927931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:40.928374 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:40.928540 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.928552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.929688 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.929691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.930522 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:40.930535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.931177 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.931183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:40.931355 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.931852 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:40.931867 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:40.933809 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:40.936930 (+ 3121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.936933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.936936 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.937262 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.937639 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:40.937644 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:40.937647 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:40.937787 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:40.938243 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:40.938247 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:40.938379 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 07:12:40.938541 (+ 162us) client_negotiation.cc:240] Negotiation successful
0812 07:12:40.938840 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:12:40.939342 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.927575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41364 (local address 127.21.181.124:44521)
0812 07:12:40.927704 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.927708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.927721 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:12:40.927756 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.927759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.927805 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.927887 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.928669 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.929568 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.930656 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.931045 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.931130 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.933968 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.933972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.933983 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.934009 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.936815 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.937378 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.937383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.937387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.937526 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.937885 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.937889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.937891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.938133 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.938256 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.938645 (+ 389us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.939009 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":46}
I20260812 07:12:40.943933 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.928781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33520 (local address 127.21.181.125:40955)
0812 07:12:40.928913 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.928917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.929061 (+ 144us) server_negotiation.cc:407] Connection header received
0812 07:12:40.929192 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.929196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.929244 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.929312 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.930517 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.931288 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.932685 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.933130 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.933240 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.938468 (+ 5228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.938473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.938477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.938519 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.941682 (+ 3163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.942285 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.942290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.942294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.942439 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.942756 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.942761 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.942764 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.943075 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.943317 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.943425 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.943644 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260812 07:12:40.944900 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: Bootstrap starting.
I20260812 07:12:40.948271 14762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:40.948275 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:40.949240 14762 log.cc:824] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:40.951810 14762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: No bootstrap required, opened a new log
I20260812 07:12:40.962715 14762 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:40.963135 14762 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:40.963260 14762 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e35a5c1f0554655a488d2a00544e22f, State: Initialized, Role: FOLLOWER
I20260812 07:12:40.964820 14762 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:40.971160 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.952739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.21.181.126:44707)
0812 07:12:40.952895 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.952900 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.953560 (+ 660us) server_negotiation.cc:407] Connection header received
0812 07:12:40.953810 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.953816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.953893 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.953959 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.956419 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.957564 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.959964 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.960688 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.960784 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.964676 (+ 3892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.964682 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.964686 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.964732 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.967963 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.968807 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.968813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.968816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.968963 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.969486 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.969491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.969494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.969824 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.970061 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.970592 (+ 531us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.970812 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:12:40.971695 14786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:40.972146 14786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:40.973894 14762 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:40.980486 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.953364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.21.181.124:44521)
0812 07:12:40.953548 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.953553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.964174 (+ 10621us) server_negotiation.cc:407] Connection header received
0812 07:12:40.964340 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.964345 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.964416 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.964522 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.966444 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.967521 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.969503 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.970153 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.970240 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.974101 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.974108 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.974112 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.974139 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.977334 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.978161 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.978166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.978169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.978340 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.978714 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.978719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.978723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.979071 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.979287 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.979890 (+ 603us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.980144 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260812 07:12:40.981377 14639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:40.953869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33522 (local address 127.21.181.125:40955)
0812 07:12:40.954010 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:40.954015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:40.965306 (+ 11291us) server_negotiation.cc:407] Connection header received
0812 07:12:40.965694 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:40.965700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:40.965774 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:40.965838 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:40.967614 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.968715 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.970862 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:40.971526 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:40.971619 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:40.974990 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:40.974997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:40.975001 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:40.975039 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:40.978174 (+ 3135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.978904 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.978909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.978912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.979057 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:40.979425 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:40.979429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:40.979431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:40.979700 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:40.979923 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:40.980994 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0812 07:12:40.981153 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260812 07:12:40.991050 14809 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:40.991173 14809 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:40.988328 14645 master_runner.cc:186] Error getting master registration for 127.21.181.126:44707: OK, instance_id { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" instance_seqno: 1786518759909996 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:12:40.996289 14645 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:12:41.038497 14501 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 4
Aug 12 07:12:41 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518759, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:12:41.052709 14503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:41.039031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41374)
0812 07:12:41.039205 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:41.039222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:41.039306 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:41.039641 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:41.039644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:41.040074 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:41.040263 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:41.040272 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.041667 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.041671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:41.042549 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:41.042560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.043467 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.043470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:41.043587 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:41.044165 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:41.044183 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:41.046322 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:41.050335 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:41.050339 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:41.050341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:41.050736 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:41.051104 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:41.051107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:41.051110 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:41.051238 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:41.051781 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:41.051784 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:41.051906 (+ 122us) client_negotiation.cc:769] Sending connection context
0812 07:12:41.052044 (+ 138us) client_negotiation.cc:240] Negotiation successful
0812 07:12:41.052331 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 07:12:41.052757 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:41.039089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.21.181.124:44521)
0812 07:12:41.039268 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:41.039274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:41.039294 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:41.039415 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:41.039420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:41.039516 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:41.039620 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:41.040390 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.041555 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.042679 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.043354 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.043473 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:41.046500 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:41.046507 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:41.046512 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:41.046549 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:41.050200 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:41.050835 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:41.050842 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:41.050846 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:41.051008 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:41.051334 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:41.051338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:41.051342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:41.051687 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:41.051834 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:41.052201 (+ 367us) server_negotiation.cc:299] Negotiation successful
0812 07:12:41.052414 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:12:41.058470 14501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: Bootstrap starting.
I20260812 07:12:41.062444 14501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:41.063735 14501 log.cc:824] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:41.066759 14501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: No bootstrap required, opened a new log
I20260812 07:12:41.070355 14786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:41.070631 14786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.072144 14786 leader_election.cc:288] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e9127771a214304a130b1d943d9f8cb (127.21.181.126:44707), d52cc5090a9a4f4495d959072930fadd (127.21.181.125:40955)
I20260812 07:12:41.075470 14501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.075824 14501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:41.075935 14501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e9127771a214304a130b1d943d9f8cb, State: Initialized, Role: FOLLOWER
I20260812 07:12:41.076556 14501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.079324 14817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.080060 14817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:41.080452 14501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:41.080515 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e35a5c1f0554655a488d2a00544e22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e9127771a214304a130b1d943d9f8cb" is_pre_election: true
I20260812 07:12:41.081084 14458 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e35a5c1f0554655a488d2a00544e22f in term 0.
I20260812 07:12:41.081717 14674 leader_election.cc:302] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e35a5c1f0554655a488d2a00544e22f, 5e9127771a214304a130b1d943d9f8cb; no voters:
I20260812 07:12:41.082139 14786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:41.082281 14786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:41.082372 14786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:41.086304 14786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.087888 14458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e35a5c1f0554655a488d2a00544e22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e9127771a214304a130b1d943d9f8cb"
I20260812 07:12:41.087988 14786 leader_election.cc:288] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [CANDIDATE]: Term 1 election: Requested vote from peers 5e9127771a214304a130b1d943d9f8cb (127.21.181.126:44707), d52cc5090a9a4f4495d959072930fadd (127.21.181.125:40955)
I20260812 07:12:41.088582 14458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:41.091650 14458 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e35a5c1f0554655a488d2a00544e22f in term 1.
I20260812 07:12:41.092068 14674 leader_election.cc:302] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e35a5c1f0554655a488d2a00544e22f, 5e9127771a214304a130b1d943d9f8cb; no voters:
I20260812 07:12:41.092366 14786 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:41.093036 14786 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [term 1 LEADER]: Becoming Leader. State: Replica: 3e35a5c1f0554655a488d2a00544e22f, State: Running, Role: LEADER
I20260812 07:12:41.093602 14786 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.108583 14634 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } attempt: 5
I20260812 07:12:41.109086 14835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e35a5c1f0554655a488d2a00544e22f. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.111402 14835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: This master's current role is: LEADER
W20260812 07:12:41.114153 14838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:41.114261 14838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 12 07:12:41 dist-test-slave-lgr5 krb5kdc[14358](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518760, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:12:41.125465 14636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:41.109089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44521 (local address 127.0.0.1:41386)
0812 07:12:41.109318 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:41.109337 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:41.109439 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:41.109959 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:41.109963 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:41.110476 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:41.110682 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:41.110695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.112325 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.112330 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:41.113662 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:41.113679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.114809 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.114815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:41.115035 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:41.115718 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:41.115742 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:41.118473 (+ 2731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:41.122514 (+ 4041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:41.122519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:41.122523 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:41.123008 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:41.123454 (+ 446us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:41.123459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:41.123462 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:41.123678 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:41.124444 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:41.124449 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:41.124611 (+ 162us) client_negotiation.cc:769] Sending connection context
0812 07:12:41.124767 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 07:12:41.125082 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260812 07:12:41.126374 14768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:41.109166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.21.181.124:44521)
0812 07:12:41.109348 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:41.109354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:41.109377 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:12:41.109594 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:41.109600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:41.109680 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:41.109761 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:41.110850 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.112154 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.113833 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:41.114500 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:41.114617 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:41.118700 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:41.118707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:41.118711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:41.118750 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:41.122354 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:41.123151 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:41.123157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:41.123161 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:41.123315 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:41.123815 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:41.123820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:41.123831 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:41.124223 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:41.124375 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:41.125755 (+ 1380us) server_negotiation.cc:299] Negotiation successful
0812 07:12:41.125994 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260812 07:12:41.115468 14681 tablet.cc:2369] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:41.134080 14634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: Bootstrap starting.
I20260812 07:12:41.149438 14634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: Neither blocks nor log segments found. Creating new log.
W20260812 07:12:41.151260 14422 tablet.cc:2369] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:41.108696 14378 master_runner.cc:186] Error getting master registration for 127.21.181.125:40955: OK, instance_id { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" instance_seqno: 1786518760389692 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:12:41.154227 14378 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:12:41.156206 14634 log.cc:824] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:41.157147 14842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:41.162988 14842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:41.167951 14634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: No bootstrap required, opened a new log
I20260812 07:12:41.183292 14634 raft_consensus.cc:348] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.183714 14634 raft_consensus.cc:374] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:41.183862 14634 raft_consensus.cc:729] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d52cc5090a9a4f4495d959072930fadd, State: Initialized, Role: FOLLOWER
I20260812 07:12:41.184597 14634 consensus_queue.cc:260] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } }
I20260812 07:12:41.185601 14849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.186007 14849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:41.186820 14634 sys_catalog.cc:564] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:41.186868 14584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e35a5c1f0554655a488d2a00544e22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52cc5090a9a4f4495d959072930fadd"
I20260812 07:12:41.186892 14585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3e35a5c1f0554655a488d2a00544e22f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d52cc5090a9a4f4495d959072930fadd" is_pre_election: true
I20260812 07:12:41.187471 14585 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e35a5c1f0554655a488d2a00544e22f in term 0.
W20260812 07:12:41.202953 14870 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:41.203090 14870 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:41.207454 14585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:41.213069 14585 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [term 1 FOLLOWER]: Refusing update from remote peer 3e35a5c1f0554655a488d2a00544e22f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:41.215693 14835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:41.224401 14458 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [term 1 FOLLOWER]: Refusing update from remote peer 3e35a5c1f0554655a488d2a00544e22f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:41.225306 14835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:41.249936 14817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e35a5c1f0554655a488d2a00544e22f. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.250257 14817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:41.260546 14817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.260779 14817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:41.261602 14847 mvcc.cc:205] Tried to move back new op lower bound from 7317580845895069696 to 7317580845444059136. Current Snapshot: MvccSnapshot[applied={T|T < 7317580845895069696}]
I20260812 07:12:41.262446 14786 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.262754 14786 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:41.264410 14842 catalog_manager.cc:1347] Generated new cluster ID: edeebd2d4b39401c8608cea5b6e495ae
I20260812 07:12:41.264503 14842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:41.264943 14849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e35a5c1f0554655a488d2a00544e22f. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.265319 14849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:41.267956 14877 mvcc.cc:205] Tried to move back new op lower bound from 7317580845895069696 to 7317580845444059136. Current Snapshot: MvccSnapshot[applied={T|T < 7317580845895069696}]
I20260812 07:12:41.275230 14835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.275596 14835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:41.276787 14849 sys_catalog.cc:455] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3e35a5c1f0554655a488d2a00544e22f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e9127771a214304a130b1d943d9f8cb" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44707 } } peers { permanent_uuid: "d52cc5090a9a4f4495d959072930fadd" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 40955 } } peers { permanent_uuid: "3e35a5c1f0554655a488d2a00544e22f" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44521 } } }
I20260812 07:12:41.277729 14849 sys_catalog.cc:458] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:12:41.303580 14842 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:41.305483 14842 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:41.322516 14842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e35a5c1f0554655a488d2a00544e22f: Generated new TSK 0
I20260812 07:12:41.323544 14842 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:42.116297 14838 catalog_manager.cc:1259] Loaded cluster ID: edeebd2d4b39401c8608cea5b6e495ae
I20260812 07:12:42.116401 14838 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: loading cluster ID for follower catalog manager: success
I20260812 07:12:42.118860 14838 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: acquiring CA information for follower catalog manager: success
I20260812 07:12:42.120647 14838 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5e9127771a214304a130b1d943d9f8cb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:12:42.205060 14870 catalog_manager.cc:1259] Loaded cluster ID: edeebd2d4b39401c8608cea5b6e495ae
I20260812 07:12:42.205173 14870 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: loading cluster ID for follower catalog manager: success
I20260812 07:12:42.207556 14870 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: acquiring CA information for follower catalog manager: success
I20260812 07:12:42.209311 14870 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d52cc5090a9a4f4495d959072930fadd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:12:42.464941 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 14378
I20260812 07:12:42.466075 14493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:42.571170 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 14378
I20260812 07:12:42.596971 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 14515
I20260812 07:12:42.598011 14618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:42.702033 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 14515
I20260812 07:12:42.728843 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 14645
I20260812 07:12:42.729912 14757 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:42.837728 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 14645
2026-08-12T07:12:42Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4842 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:42 dist-test-slave-lgr5 krb5kdc[14904](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:42 dist-test-slave-lgr5 krb5kdc[14904](info): set up 2 sockets
Aug 12 07:12:42 dist-test-slave-lgr5 krb5kdc[14904](info): commencing operation
krb5kdc: starting...
W20260812 07:12:44.093030 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.002s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:12:44 dist-test-slave-lgr5 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:12:44.329165 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34731
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:36305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:34731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:44.654258 14928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:44.654543 14928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:44.654695 14928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:44.661893 14928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:44.662034 14928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:44.662122 14928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:44.662206 14928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:44.662276 14928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:44.674923 14928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:34731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:34731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:44.676141 14928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:44.677434 14928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:44.688650 14943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:44.688612 14939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:44.688695 14937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:44.689831 14928 server_base.cc:1034] running on GCE node
I20260812 07:12:44.690513 14928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:44.691831 14928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:44.693044 14928 hybrid_clock.cc:648] HybridClock initialized: now 1786518764692961 us; error 71 us; skew 500 ppm
I20260812 07:12:44.693358 14928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:44 dist-test-slave-lgr5 krb5kdc[14904](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518764, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:44.696228 14928 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:44.697548 14928 webserver.cc:460] Webserver started at http://127.21.181.126:39269/ using document root <none> and password file <none>
I20260812 07:12:44.698199 14928 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:44.698315 14928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:44.698597 14928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:44.701470 14928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "7b9bf163c92947db88e30a2e0e415cf5"
format_stamp: "Formatted at 2026-08-12 07:12:44 on dist-test-slave-lgr5"
server_key: "392639a9b63f670070f080fdd16cdb26"
server_key_iv: "a900924f0a544a501549d63e6ea53618"
server_key_version: "encryptionkey@0"
I20260812 07:12:44.702212 14928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "7b9bf163c92947db88e30a2e0e415cf5"
format_stamp: "Formatted at 2026-08-12 07:12:44 on dist-test-slave-lgr5"
server_key: "392639a9b63f670070f080fdd16cdb26"
server_key_iv: "a900924f0a544a501549d63e6ea53618"
server_key_version: "encryptionkey@0"
I20260812 07:12:44.706980 14928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260812 07:12:44.710165 14956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:44.711313 14928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:12:44.711457 14928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "7b9bf163c92947db88e30a2e0e415cf5"
format_stamp: "Formatted at 2026-08-12 07:12:44 on dist-test-slave-lgr5"
server_key: "392639a9b63f670070f080fdd16cdb26"
server_key_iv: "a900924f0a544a501549d63e6ea53618"
server_key_version: "encryptionkey@0"
I20260812 07:12:44.711627 14928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:44.757545 14928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:44.761179 14928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:44.761430 14928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:44.778098 14928 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:34731
I20260812 07:12:44.778117 15047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:34731 every 8 connection(s)
I20260812 07:12:44.779336 14928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:44.782811 15048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:44.789265 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 14928
I20260812 07:12:44.789466 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:12:44.790110 22229 external_mini_cluster.cc:1428] Setting key 130c13839c154d2a5adaaad7fb46f10c
I20260812 07:12:44.792860 15048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5: Bootstrap starting.
I20260812 07:12:44.795759 15048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:44.796773 15048 log.cc:824] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:44.799329 15048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5: No bootstrap required, opened a new log
Aug 12 07:12:44 dist-test-slave-lgr5 krb5kdc[14904](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:44.807168 15048 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b9bf163c92947db88e30a2e0e415cf5" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34731 } }
I20260812 07:12:44.807503 15048 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:44.807590 15048 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b9bf163c92947db88e30a2e0e415cf5, State: Initialized, Role: FOLLOWER
I20260812 07:12:44.808197 15048 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b9bf163c92947db88e30a2e0e415cf5" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34731 } }
I20260812 07:12:44.808485 15048 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:44.808619 15048 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:44.808789 15048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:44.811694 15054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:44.791806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.21.181.126:34731)
0812 07:12:44.792584 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:44.792594 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:44.792632 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:12:44.794319 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:44.794340 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:44.794689 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:44.795018 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:44.796588 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:44.798187 (+ 1599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:44.799487 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:44.800157 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:44.800309 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:44.804380 (+ 4071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:44.804392 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:44.804405 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:44.804443 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:44.807292 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:44.808045 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:44.808053 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:44.808061 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:44.808296 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:44.808760 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:44.808765 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:44.808768 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:44.809448 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:44.809755 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:44.810075 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 07:12:44.810482 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":370,"threads_started":1}
I20260812 07:12:44.811843 15048 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b9bf163c92947db88e30a2e0e415cf5" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34731 } }
I20260812 07:12:44.812348 15048 leader_election.cc:302] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b9bf163c92947db88e30a2e0e415cf5; no voters:
I20260812 07:12:44.812901 15048 leader_election.cc:288] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:44.813032 15057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:44.815821 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:12:44.816370 15057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [term 1 LEADER]: Becoming Leader. State: Replica: 7b9bf163c92947db88e30a2e0e415cf5, State: Running, Role: LEADER
I20260812 07:12:44.816812 15057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b9bf163c92947db88e30a2e0e415cf5" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34731 } }
I20260812 07:12:44.817714 15048 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:44.821805 15059 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b9bf163c92947db88e30a2e0e415cf5. Latest consensus state: current_term: 1 leader_uuid: "7b9bf163c92947db88e30a2e0e415cf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b9bf163c92947db88e30a2e0e415cf5" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34731 } } }
I20260812 07:12:44.821835 15058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b9bf163c92947db88e30a2e0e415cf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b9bf163c92947db88e30a2e0e415cf5" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 34731 } } }
I20260812 07:12:44.822237 15058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:44.822237 15059 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5 [sys.catalog]: This master's current role is: LEADER
W20260812 07:12:44.827802 15085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:12:44.827913 15085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:12:44.828027 15075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:44.829608 15075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:44.836372 15075 catalog_manager.cc:1347] Generated new cluster ID: 17e0e8b6835142a89dfaf722cd87c199
I20260812 07:12:44.836473 15075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:44.849807 15075 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:44.850947 15075 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:44.858244 15075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b9bf163c92947db88e30a2e0e415cf5: Generated new TSK 0
I20260812 07:12:44.858906 15075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:12:44.876854 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 14928
I20260812 07:12:44.877938 15040 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:44.965394 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 14928
2026-08-12T07:12:44Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2124 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:12:45 dist-test-slave-lgr5 krb5kdc[15098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:12:45 dist-test-slave-lgr5 krb5kdc[15098](info): set up 2 sockets
Aug 12 07:12:45 dist-test-slave-lgr5 krb5kdc[15098](info): commencing operation
krb5kdc: starting...
W20260812 07:12:46.218878 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:12:46 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:12:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:12:46.455363 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33479
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:46.770139 15123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:46.770390 15123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:46.770481 15123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:46.777333 15123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:46.777423 15123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:46.777482 15123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:46.777536 15123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:46.777580 15123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:46.786651 15123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:33479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:46.787760 15123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:46.788947 15123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:46.799741 15137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:46.799713 15141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:46.799676 15138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:46.800514 15123 server_base.cc:1034] running on GCE node
I20260812 07:12:46.801172 15123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:46.802407 15123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:46.803612 15123 hybrid_clock.cc:648] HybridClock initialized: now 1786518766803561 us; error 45 us; skew 500 ppm
I20260812 07:12:46.803918 15123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:46 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518766, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:46.806809 15123 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:46.808207 15123 webserver.cc:460] Webserver started at http://127.21.181.126:35691/ using document root <none> and password file <none>
I20260812 07:12:46.808827 15123 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:46.808943 15123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:46.809226 15123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:46.811961 15123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "b06169228d9a4eb7886295daa107969c"
format_stamp: "Formatted at 2026-08-12 07:12:46 on dist-test-slave-lgr5"
server_key: "9e7d06e7aa2d368b197b2cf107ed1e11"
server_key_iv: "935f8177cba5fdd254054184b9c9f853"
server_key_version: "encryptionkey@0"
I20260812 07:12:46.812738 15123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "b06169228d9a4eb7886295daa107969c"
format_stamp: "Formatted at 2026-08-12 07:12:46 on dist-test-slave-lgr5"
server_key: "9e7d06e7aa2d368b197b2cf107ed1e11"
server_key_iv: "935f8177cba5fdd254054184b9c9f853"
server_key_version: "encryptionkey@0"
I20260812 07:12:46.817458 15123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:12:46.821736 15161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:46.822932 15123 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:12:46.823081 15123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "b06169228d9a4eb7886295daa107969c"
format_stamp: "Formatted at 2026-08-12 07:12:46 on dist-test-slave-lgr5"
server_key: "9e7d06e7aa2d368b197b2cf107ed1e11"
server_key_iv: "935f8177cba5fdd254054184b9c9f853"
server_key_version: "encryptionkey@0"
I20260812 07:12:46.823246 15123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:46.846781 15123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:46.850339 15123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:46.850602 15123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:46.867513 15123 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:33479
I20260812 07:12:46.867542 15244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:33479 every 8 connection(s)
I20260812 07:12:46.868759 15123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:46.872177 15245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:46.874257 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 15123
I20260812 07:12:46.874447 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:12:46.875124 22229 external_mini_cluster.cc:1428] Setting key b4572ccd80071ca1335106db2dc7343b
I20260812 07:12:46.884343 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap starting.
I20260812 07:12:46.887637 15245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:46.888619 15245 log.cc:824] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:12:46 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:46.891505 15245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: No bootstrap required, opened a new log
I20260812 07:12:46.898022 15249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:46.877178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.21.181.126:33479)
0812 07:12:46.877980 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:46.878004 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:46.878366 (+ 362us) server_negotiation.cc:407] Connection header received
0812 07:12:46.880328 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:46.880356 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:46.880825 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:46.881285 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:46.882507 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:46.884449 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:46.885707 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:46.886708 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:46.886895 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:46.891044 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:46.891053 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:46.891065 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:46.891099 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:46.894098 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:46.894912 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:46.894918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:46.894924 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:46.895109 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:46.895495 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:46.895497 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:46.895499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:46.896007 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:46.896294 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:46.896672 (+ 378us) server_negotiation.cc:299] Negotiation successful
0812 07:12:46.896998 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":348,"threads_started":1}
I20260812 07:12:46.900023 15245 raft_consensus.cc:348] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:46.900393 15245 raft_consensus.cc:374] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:46.900498 15245 raft_consensus.cc:729] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06169228d9a4eb7886295daa107969c, State: Initialized, Role: FOLLOWER
I20260812 07:12:46.900997 15245 consensus_queue.cc:260] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:46.901209 15245 raft_consensus.cc:388] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:46.901329 15245 raft_consensus.cc:482] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:46.901477 15245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:46.903616 15245 raft_consensus.cc:504] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:46.904067 15245 leader_election.cc:302] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b06169228d9a4eb7886295daa107969c; no voters:
I20260812 07:12:46.904649 15245 leader_election.cc:288] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:12:46.904789 15251 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:46.905606 15251 raft_consensus.cc:686] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 LEADER]: Becoming Leader. State: Replica: b06169228d9a4eb7886295daa107969c, State: Running, Role: LEADER
I20260812 07:12:46.906126 15251 consensus_queue.cc:237] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:46.906569 15245 sys_catalog.cc:564] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:46.908938 15256 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b06169228d9a4eb7886295daa107969c. Latest consensus state: current_term: 1 leader_uuid: "b06169228d9a4eb7886295daa107969c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } } }
I20260812 07:12:46.908993 15254 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b06169228d9a4eb7886295daa107969c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } } }
I20260812 07:12:46.909341 15254 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:46.909343 15256 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:46.919391 15264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:46.923554 15264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:46.931155 15264 catalog_manager.cc:1347] Generated new cluster ID: ea5e08c63b3e4caeb3ce23f3bdff762b
I20260812 07:12:46.931264 15264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:46.943944 15264 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:12:46.945685 15264 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:46.956629 15264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Generated new TSK 0
I20260812 07:12:46.957515 15264 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:12:47.018996 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:33479
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:47.338538 15297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:47.338809 15297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:47.338923 15297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:47.345762 15297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:47.345858 15297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:47.345969 15297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:12:47.354972 15297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:47.356043 15297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:47.357257 15297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:47.369563 15304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:47.369597 15305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:47.369626 15307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:47.370291 15297 server_base.cc:1034] running on GCE node
I20260812 07:12:47.370844 15297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:47.371686 15297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:47.372879 15297 hybrid_clock.cc:648] HybridClock initialized: now 1786518767372835 us; error 31 us; skew 500 ppm
I20260812 07:12:47.373162 15297 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:47 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:47.376057 15297 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:12:47.377352 15297 webserver.cc:460] Webserver started at http://127.21.181.65:34423/ using document root <none> and password file <none>
I20260812 07:12:47.377974 15297 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:47.378090 15297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:47.378365 15297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:47.381153 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "70c73d88af35ef6dcf04e1082a0db883"
server_key_iv: "24a2c42cb89d6188f45b93f15024d9f0"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.381904 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "70c73d88af35ef6dcf04e1082a0db883"
server_key_iv: "24a2c42cb89d6188f45b93f15024d9f0"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.386579 15297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:12:47.389739 15319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.390923 15297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:12:47.391073 15297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "70c73d88af35ef6dcf04e1082a0db883"
server_key_iv: "24a2c42cb89d6188f45b93f15024d9f0"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.391245 15297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:47.407621 15297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:47.411216 15297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:47.411459 15297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:47.412343 15297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:47.413602 15297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:47.413679 15297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.413794 15297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:47.413846 15297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.443396 15297 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:40413
I20260812 07:12:47.443445 15499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:40413 every 8 connection(s)
I20260812 07:12:47.444607 15297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:47.450668 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 15297
I20260812 07:12:47.450853 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:12:47.451506 22229 external_mini_cluster.cc:1428] Setting key 5aed17a2851fc547e52ecb22002792a9
Aug 12 07:12:47 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:47.463671 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:47.446896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:38949 (local address 127.21.181.126:33479)
0812 07:12:47.447309 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:47.447314 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:47.448229 (+ 915us) server_negotiation.cc:407] Connection header received
0812 07:12:47.449923 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:47.449927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:47.449992 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:47.450084 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:47.451521 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.452519 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.454100 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.454633 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.454754 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:47.458417 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:47.458423 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:47.458426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:47.458458 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:47.460916 (+ 2458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:47.461678 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:47.461681 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:47.461683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:47.461792 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:47.462210 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:47.462213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:47.462214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:47.462455 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:47.462560 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:47.463254 (+ 694us) server_negotiation.cc:299] Negotiation successful
0812 07:12:47.463414 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":209,"threads_started":1}
I20260812 07:12:47.464875 15506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:47.447386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:38949)
0812 07:12:47.448055 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:47.448126 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:47.449696 (+ 1570us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:47.450263 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:47.450289 (+ 26us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:47.450767 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:47.451338 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:47.451355 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.452673 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.452677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:47.453946 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:47.453959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.454810 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.454815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:47.455018 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:47.455608 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:47.455630 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:47.458211 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:47.461047 (+ 2836us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:47.461054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:47.461068 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:47.461587 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:47.461885 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:47.461889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:47.461892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:47.462124 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:47.462567 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:47.462575 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:47.462936 (+ 361us) client_negotiation.cc:769] Sending connection context
0812 07:12:47.463217 (+ 281us) client_negotiation.cc:240] Negotiation successful
0812 07:12:47.463675 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":250,"threads_started":1}
I20260812 07:12:47.466336 15500 heartbeater.cc:347] Connected to a master server at 127.21.181.126:33479
I20260812 07:12:47.466642 15500 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:47.467456 15500 heartbeater.cc:511] Master 127.21.181.126:33479 requested a full tablet report, sending...
I20260812 07:12:47.469362 15190 ts_manager.cc:194] Registered new tserver with Master: f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413)
I20260812 07:12:47.470738 15190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:38949
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:12:47.508543 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:33479
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:47.820883 15509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:47.821139 15509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:47.821280 15509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:47.828190 15509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:47.828294 15509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:47.828439 15509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:12:47.837446 15509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:47.838557 15509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:47.839740 15509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:47.851645 15519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:47.851725 15523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:47.851691 15521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:47.852692 15509 server_base.cc:1034] running on GCE node
I20260812 07:12:47.853263 15509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:47.854080 15509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:47.855274 15509 hybrid_clock.cc:648] HybridClock initialized: now 1786518767855218 us; error 44 us; skew 500 ppm
I20260812 07:12:47.855556 15509 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:47 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:47.858522 15509 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:12:47.859784 15509 webserver.cc:460] Webserver started at http://127.21.181.66:37343/ using document root <none> and password file <none>
I20260812 07:12:47.860448 15509 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:47.860566 15509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:47.860833 15509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:47.863523 15509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "1fb26d543d1f4d4191403a9819b68151"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "cb7a516c1b806ddfd6ec6f17b19a976d"
server_key_iv: "6b45d2277779abbf9d14dc87b8cfbf72"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.864302 15509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "1fb26d543d1f4d4191403a9819b68151"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "cb7a516c1b806ddfd6ec6f17b19a976d"
server_key_iv: "6b45d2277779abbf9d14dc87b8cfbf72"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.869575 15509 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260812 07:12:47.873579 15535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.874965 15509 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:12:47.875144 15509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1fb26d543d1f4d4191403a9819b68151"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "cb7a516c1b806ddfd6ec6f17b19a976d"
server_key_iv: "6b45d2277779abbf9d14dc87b8cfbf72"
server_key_version: "encryptionkey@0"
I20260812 07:12:47.875347 15509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:47.889648 15509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:47.892970 15509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:47.893215 15509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:47.894008 15509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:47.895190 15509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:47.895258 15509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.895340 15509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:47.895381 15509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:47.924752 15509 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:34253
I20260812 07:12:47.924849 15746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:34253 every 8 connection(s)
I20260812 07:12:47.925894 15509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:47.928578 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 15509
I20260812 07:12:47.928740 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:12:47.929255 22229 external_mini_cluster.cc:1428] Setting key e1507b4631aa47f5fcc6453d9bb0bd47
Aug 12 07:12:47 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518767, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:47.946167 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:47.928370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:38753 (local address 127.21.181.126:33479)
0812 07:12:47.928586 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:47.928592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:47.929874 (+ 1282us) server_negotiation.cc:407] Connection header received
0812 07:12:47.932102 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:47.932108 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:47.932178 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:47.932252 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:47.934086 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.934795 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.936828 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.937219 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.937307 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:47.940970 (+ 3663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:47.940975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:47.940978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:47.941011 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:47.943291 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:47.943960 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:47.943964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:47.943967 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:47.944117 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:47.944542 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:47.944546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:47.944549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:47.944799 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:47.944943 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:47.945597 (+ 654us) server_negotiation.cc:299] Negotiation successful
0812 07:12:47.945836 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:12:47.947005 15751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:47.928811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:38753)
0812 07:12:47.929711 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:47.929758 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:47.931793 (+ 2035us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:47.932517 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:47.932532 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:47.933178 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:47.933936 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:47.933961 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.934942 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.934946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:47.936707 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:47.936724 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:47.937425 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:47.937432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:47.937706 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:47.938367 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:47.938394 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:47.940762 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:47.943420 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:47.943429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:47.943444 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:47.943839 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:47.944230 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:47.944235 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:47.944238 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:47.944418 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:47.944926 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:47.944934 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:47.945237 (+ 303us) client_negotiation.cc:769] Sending connection context
0812 07:12:47.945458 (+ 221us) client_negotiation.cc:240] Negotiation successful
0812 07:12:47.945949 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":321,"threads_started":1}
I20260812 07:12:47.948359 15747 heartbeater.cc:347] Connected to a master server at 127.21.181.126:33479
I20260812 07:12:47.948694 15747 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:47.949565 15747 heartbeater.cc:511] Master 127.21.181.126:33479 requested a full tablet report, sending...
I20260812 07:12:47.951108 15190 ts_manager.cc:194] Registered new tserver with Master: 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66:34253)
I20260812 07:12:47.952147 15190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:38753
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:12:47.988504 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:33479
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:48.302533 15757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:48.302770 15757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:48.302897 15757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:48.309938 15757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:48.310036 15757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:48.310187 15757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:12:48.319635 15757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:48.320714 15757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:48.321915 15757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:48.333717 15765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:48.333650 15771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:48.333748 15766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:48.334769 15757 server_base.cc:1034] running on GCE node
I20260812 07:12:48.335279 15757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:48.336108 15757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:48.337368 15757 hybrid_clock.cc:648] HybridClock initialized: now 1786518768337285 us; error 45 us; skew 500 ppm
I20260812 07:12:48.337672 15757 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:48 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518768, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:48.340718 15757 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:12:48.341997 15757 webserver.cc:460] Webserver started at http://127.21.181.67:36495/ using document root <none> and password file <none>
I20260812 07:12:48.342612 15757 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:48.342730 15757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:48.343010 15757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:12:48.345806 15757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "8125d793dc11415ebcb7d465e3545ccb"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-lgr5"
server_key: "64448239d02f1ca1a3c87ec209a44129"
server_key_iv: "965558ddfb393687e95b08e388a5d147"
server_key_version: "encryptionkey@0"
I20260812 07:12:48.346555 15757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "8125d793dc11415ebcb7d465e3545ccb"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-lgr5"
server_key: "64448239d02f1ca1a3c87ec209a44129"
server_key_iv: "965558ddfb393687e95b08e388a5d147"
server_key_version: "encryptionkey@0"
I20260812 07:12:48.351225 15757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:12:48.354425 15787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:48.355619 15757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:12:48.355758 15757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "8125d793dc11415ebcb7d465e3545ccb"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-lgr5"
server_key: "64448239d02f1ca1a3c87ec209a44129"
server_key_iv: "965558ddfb393687e95b08e388a5d147"
server_key_version: "encryptionkey@0"
I20260812 07:12:48.355924 15757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:48.372251 15757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:48.375833 15757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:48.376071 15757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:48.376911 15757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:48.378134 15757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:48.378206 15757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:48.378316 15757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:48.378365 15757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:48.405571 15757 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:33957
I20260812 07:12:48.405606 15997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:33957 every 8 connection(s)
I20260812 07:12:48.406705 15757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:48.409291 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 15757
I20260812 07:12:48.409475 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:12:48.410055 22229 external_mini_cluster.cc:1428] Setting key 4e6ea813fa05368b89e254e8238e6b03
Aug 12 07:12:48 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518768, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:48.425817 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.409097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:42571 (local address 127.21.181.126:33479)
0812 07:12:48.409262 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.409267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.410591 (+ 1324us) server_negotiation.cc:407] Connection header received
0812 07:12:48.412754 (+ 2163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.412759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.412842 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.412930 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:48.414465 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.415230 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.416783 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.417202 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.417312 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.420428 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:48.420433 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:48.420437 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:48.420469 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:48.422917 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:48.423586 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:48.423590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:48.423593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:48.423711 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:48.424190 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:48.424195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:48.424197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:48.424470 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:48.424611 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.425247 (+ 636us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.425491 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:12:48.426613 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.409682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:42571)
0812 07:12:48.410440 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:48.410486 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:48.412514 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:48.413201 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:48.413213 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:48.413781 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:48.414305 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.414322 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.415380 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.415386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.416646 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.416662 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.417418 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.417426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.417664 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.418232 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:48.418258 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:48.420237 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:48.423051 (+ 2814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:48.423059 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:48.423073 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:48.423443 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:48.423814 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:48.423819 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:48.423823 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:48.424066 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:48.424604 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:48.424612 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:48.424900 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 07:12:48.425130 (+ 230us) client_negotiation.cc:240] Negotiation successful
0812 07:12:48.425573 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":312,"threads_started":1}
I20260812 07:12:48.427748 15999 heartbeater.cc:347] Connected to a master server at 127.21.181.126:33479
I20260812 07:12:48.428030 15999 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:48.428716 15999 heartbeater.cc:511] Master 127.21.181.126:33479 requested a full tablet report, sending...
I20260812 07:12:48.430040 15190 ts_manager.cc:194] Registered new tserver with Master: 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67:33957)
I20260812 07:12:48.430796 15190 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:42571
I20260812 07:12:48.437305 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:48.453294 15502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.441437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46754 (local address 127.21.181.126:33479)
0812 07:12:48.441589 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.441593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.441995 (+ 402us) server_negotiation.cc:407] Connection header received
0812 07:12:48.442130 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.442133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.442185 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.442259 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:48.443630 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.444765 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.446068 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.446731 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.446863 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.449113 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:48.449118 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:48.449122 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:48.449151 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:48.451255 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:48.451801 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:48.451804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:48.451806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:48.451921 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:48.452249 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:48.452252 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:48.452254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:48.452538 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:48.452709 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.452946 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.453090 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:12:48.456658 15190 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:48.463218 15190 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:48.473493 15500 heartbeater.cc:503] Master 127.21.181.126:33479 was elected leader, sending a full tablet report...
I20260812 07:12:48.485025 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.474150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:34253 (local address 127.0.0.1:33820)
0812 07:12:48.475108 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:48.475148 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:48.475305 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:48.476507 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:48.476516 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:48.476549 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:48.476944 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.476957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.479064 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.479069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.481694 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.481717 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.483979 (+ 2262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.483986 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.484219 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.484273 (+ 54us) client_negotiation.cc:769] Sending connection context
0812 07:12:48.484423 (+ 150us) client_negotiation.cc:240] Negotiation successful
0812 07:12:48.484620 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":505,"threads_started":1}
I20260812 07:12:48.486088 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.474288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.21.181.66:34253)
0812 07:12:48.475603 (+ 1315us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.475612 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.475644 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:12:48.475741 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.475748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.475990 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.476206 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:48.477145 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.478884 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.481905 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.483468 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.483607 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.483815 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.485534 (+ 1719us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.485734 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1144,"thread_start_us":1027,"threads_started":1}
I20260812 07:12:48.492292 15611 tablet_service.cc:1467] Processing CreateTablet for tablet 74f1c8b67de14a0aae5c2bf8be9ab964 (DEFAULT_TABLE table=test-table [id=655bbec0fc764c6398b278c9ae7b4d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:48.493049 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.475971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:33957 (local address 127.0.0.1:37750)
0812 07:12:48.485234 (+ 9263us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:48.485249 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:48.485364 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:48.487525 (+ 2161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:48.487530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:48.487561 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:48.487742 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.487751 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.489494 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.489499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.490645 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.490686 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.492253 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.492258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.492491 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.492520 (+ 29us) client_negotiation.cc:769] Sending connection context
0812 07:12:48.492598 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:12:48.492737 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9186,"thread_start_us":11737,"threads_started":1}
I20260812 07:12:48.493117 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.476599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37750 (local address 127.21.181.67:33957)
0812 07:12:48.486891 (+ 10292us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.486900 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.486937 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:12:48.487039 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.487047 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.487292 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.487504 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:48.487950 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.489351 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.490862 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.492107 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.492227 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.492450 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.492656 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.492807 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10110,"thread_start_us":9921,"threads_started":1}
I20260812 07:12:48.494010 15611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74f1c8b67de14a0aae5c2bf8be9ab964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:48.495884 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.475855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46856 (local address 127.21.181.65:40413)
0812 07:12:48.488188 (+ 12333us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.488196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.488222 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:12:48.488306 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.488313 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.488535 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.488707 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:48.489311 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.490825 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.493215 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.494656 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.494796 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.495011 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.495359 (+ 348us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.495513 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12148,"thread_start_us":12025,"threads_started":1}
I20260812 07:12:48.496188 16023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.475409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:46856)
0812 07:12:48.475890 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:48.475906 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:48.476023 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:48.488738 (+ 12715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:48.488743 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:48.488769 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:48.489120 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.489130 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.490985 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.490990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.493029 (+ 2039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.493047 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.494847 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.494853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.495037 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.495085 (+ 48us) client_negotiation.cc:769] Sending connection context
0812 07:12:48.495177 (+ 92us) client_negotiation.cc:240] Negotiation successful
0812 07:12:48.495865 (+ 688us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":231,"threads_started":1}
I20260812 07:12:48.496672 15875 tablet_service.cc:1467] Processing CreateTablet for tablet 74f1c8b67de14a0aae5c2bf8be9ab964 (DEFAULT_TABLE table=test-table [id=655bbec0fc764c6398b278c9ae7b4d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:48.497977 15875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74f1c8b67de14a0aae5c2bf8be9ab964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:48.499823 15390 tablet_service.cc:1467] Processing CreateTablet for tablet 74f1c8b67de14a0aae5c2bf8be9ab964 (DEFAULT_TABLE table=test-table [id=655bbec0fc764c6398b278c9ae7b4d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:48.501216 15390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 74f1c8b67de14a0aae5c2bf8be9ab964. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:48.505991 16029 tablet_bootstrap.cc:492] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Bootstrap starting.
I20260812 07:12:48.509315 16029 tablet_bootstrap.cc:654] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:48.510715 16029 log.cc:824] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:48.513367 16029 tablet_bootstrap.cc:492] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: No bootstrap required, opened a new log
I20260812 07:12:48.513598 16029 ts_tablet_manager.cc:1397] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.004s
I20260812 07:12:48.514473 16033 tablet_bootstrap.cc:492] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Bootstrap starting.
I20260812 07:12:48.515125 16032 tablet_bootstrap.cc:492] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Bootstrap starting.
I20260812 07:12:48.517907 16033 tablet_bootstrap.cc:654] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:48.518175 16032 tablet_bootstrap.cc:654] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:48.519090 16032 log.cc:824] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:48.519083 16033 log.cc:824] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:48.521670 16033 tablet_bootstrap.cc:492] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: No bootstrap required, opened a new log
I20260812 07:12:48.521898 16033 ts_tablet_manager.cc:1397] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260812 07:12:48.521936 16032 tablet_bootstrap.cc:492] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: No bootstrap required, opened a new log
I20260812 07:12:48.522166 16032 ts_tablet_manager.cc:1397] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260812 07:12:48.523408 16029 raft_consensus.cc:348] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.523823 16029 raft_consensus.cc:374] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:48.523916 16029 raft_consensus.cc:729] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26d543d1f4d4191403a9819b68151, State: Initialized, Role: FOLLOWER
I20260812 07:12:48.524554 16029 consensus_queue.cc:260] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.525686 15747 heartbeater.cc:503] Master 127.21.181.126:33479 was elected leader, sending a full tablet report...
I20260812 07:12:48.526316 16029 ts_tablet_manager.cc:1428] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260812 07:12:48.531234 16032 raft_consensus.cc:348] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.531565 16032 raft_consensus.cc:374] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:48.531360 16033 raft_consensus.cc:348] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.531679 16032 raft_consensus.cc:729] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8125d793dc11415ebcb7d465e3545ccb, State: Initialized, Role: FOLLOWER
I20260812 07:12:48.531737 16033 raft_consensus.cc:374] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:48.531833 16033 raft_consensus.cc:729] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2af8fcf4ae442ec8032cb5329a4f132, State: Initialized, Role: FOLLOWER
I20260812 07:12:48.532274 16032 consensus_queue.cc:260] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.532339 16033 consensus_queue.cc:260] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.533188 15999 heartbeater.cc:503] Master 127.21.181.126:33479 was elected leader, sending a full tablet report...
I20260812 07:12:48.533771 16032 ts_tablet_manager.cc:1428] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260812 07:12:48.533900 16033 ts_tablet_manager.cc:1428] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260812 07:12:48.658210 16000 tablet.cc:2369] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:12:48.677762 15748 tablet.cc:2369] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:12:48.696610 15501 tablet.cc:2369] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:48.697855 16039 raft_consensus.cc:482] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:48.698108 16039 raft_consensus.cc:504] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.699702 16039 leader_election.cc:288] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66:34253), f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413)
I20260812 07:12:48.705495 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.700186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:52015 (local address 127.21.181.66:34253)
0812 07:12:48.700360 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.700365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.700385 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:48.700547 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.700552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.700631 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.700731 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:48.701291 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.702302 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.703840 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.704672 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.704810 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.704883 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.705130 (+ 247us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.705269 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:12:48.705515 16001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.700037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:34253 (local address 127.21.181.67:52015)
0812 07:12:48.700260 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:48.700281 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:48.700402 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:48.700781 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:48.700786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:48.700822 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:48.701108 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.701122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.702429 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.702433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.703673 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.703686 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.704811 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.704817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.704983 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.705010 (+ 27us) client_negotiation.cc:769] Sending connection context
0812 07:12:48.705091 (+ 81us) client_negotiation.cc:240] Negotiation successful
0812 07:12:48.705216 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:12:48.706203 16042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.700472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.21.181.67:55639)
0812 07:12:48.701367 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:48.701384 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:48.701522 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:48.701784 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:48.701789 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:48.701813 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:48.702137 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.702149 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.703094 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.703099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.704455 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:48.704468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.705584 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.705590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:48.705741 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.705772 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:12:48.705860 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:12:48.705950 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"thread_start_us":534,"threads_started":1}
I20260812 07:12:48.706235 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.700696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:55639 (local address 127.21.181.65:40413)
0812 07:12:48.700891 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.700896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.701468 (+ 572us) server_negotiation.cc:407] Connection header received
0812 07:12:48.701628 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.701632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.701689 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.701752 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:48.702261 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.702931 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.704608 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.705359 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.705500 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.705573 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.705895 (+ 322us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.706032 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260812 07:12:48.706655 15657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74f1c8b67de14a0aae5c2bf8be9ab964" candidate_uuid: "8125d793dc11415ebcb7d465e3545ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fb26d543d1f4d4191403a9819b68151" is_pre_election: true
I20260812 07:12:48.707185 15657 raft_consensus.cc:2454] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8125d793dc11415ebcb7d465e3545ccb in term 0.
I20260812 07:12:48.707235 15425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74f1c8b67de14a0aae5c2bf8be9ab964" candidate_uuid: "8125d793dc11415ebcb7d465e3545ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" is_pre_election: true
I20260812 07:12:48.707765 15425 raft_consensus.cc:2454] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8125d793dc11415ebcb7d465e3545ccb in term 0.
I20260812 07:12:48.707845 15793 leader_election.cc:302] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26d543d1f4d4191403a9819b68151, 8125d793dc11415ebcb7d465e3545ccb; no voters:
I20260812 07:12:48.708549 16039 raft_consensus.cc:2781] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:48.708664 16039 raft_consensus.cc:482] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:48.708729 16039 raft_consensus.cc:3037] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:48.710800 16039 raft_consensus.cc:504] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.711515 16039 leader_election.cc:288] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [CANDIDATE]: Term 1 election: Requested vote from peers 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66:34253), f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413)
I20260812 07:12:48.711947 15425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74f1c8b67de14a0aae5c2bf8be9ab964" candidate_uuid: "8125d793dc11415ebcb7d465e3545ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
I20260812 07:12:48.712010 15657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "74f1c8b67de14a0aae5c2bf8be9ab964" candidate_uuid: "8125d793dc11415ebcb7d465e3545ccb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fb26d543d1f4d4191403a9819b68151"
I20260812 07:12:48.712144 15425 raft_consensus.cc:3037] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:48.712204 15657 raft_consensus.cc:3037] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:48.714211 15425 raft_consensus.cc:2454] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8125d793dc11415ebcb7d465e3545ccb in term 1.
I20260812 07:12:48.714277 15657 raft_consensus.cc:2454] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8125d793dc11415ebcb7d465e3545ccb in term 1.
I20260812 07:12:48.714569 15793 leader_election.cc:302] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8125d793dc11415ebcb7d465e3545ccb, f2af8fcf4ae442ec8032cb5329a4f132; no voters:
I20260812 07:12:48.714800 16039 raft_consensus.cc:2781] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:48.715224 16039 raft_consensus.cc:686] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 1 LEADER]: Becoming Leader. State: Replica: 8125d793dc11415ebcb7d465e3545ccb, State: Running, Role: LEADER
I20260812 07:12:48.715651 16039 consensus_queue.cc:237] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } }
I20260812 07:12:48.720034 15190 catalog_manager.cc:5495] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67). New cstate: current_term: 1 leader_uuid: "8125d793dc11415ebcb7d465e3545ccb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } health_report { overall_health: HEALTHY } } }
I20260812 07:12:48.787569 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.782222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37756 (local address 127.21.181.67:33957)
0812 07:12:48.782413 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.782419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.782502 (+ 83us) server_negotiation.cc:407] Connection header received
0812 07:12:48.782716 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.782720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.782797 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.782927 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:48.783319 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.784026 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.785510 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.785901 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.786017 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.786519 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:48.786999 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:48.787138 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.787240 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.787320 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:12:48.795039 15425 raft_consensus.cc:1261] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Refusing update from remote peer 8125d793dc11415ebcb7d465e3545ccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:48.795140 15657 raft_consensus.cc:1261] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Refusing update from remote peer 8125d793dc11415ebcb7d465e3545ccb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:48.795957 16043 consensus_queue.cc:1035] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:48.796334 16039 consensus_queue.cc:1035] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:48.814599 16050 mvcc.cc:205] Tried to move back new op lower bound from 7317580876975288320 to 7317580876662722560. Current Snapshot: MvccSnapshot[applied={T|T < 7317580876975288320}]
I20260812 07:12:48.823927 15189 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:48.824285 15189 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:48.827659 15189 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Sending DeleteTablet for 3 replicas of tablet 74f1c8b67de14a0aae5c2bf8be9ab964
I20260812 07:12:48.828655 15611 tablet_service.cc:1514] Processing DeleteTablet for tablet 74f1c8b67de14a0aae5c2bf8be9ab964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:48 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:33820
I20260812 07:12:48.828845 15875 tablet_service.cc:1514] Processing DeleteTablet for tablet 74f1c8b67de14a0aae5c2bf8be9ab964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:48 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:37750
I20260812 07:12:48.829182 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 15297
I20260812 07:12:48.829321 15390 tablet_service.cc:1514] Processing DeleteTablet for tablet 74f1c8b67de14a0aae5c2bf8be9ab964 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:48 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:46856
I20260812 07:12:48.830124 16067 tablet_replica.cc:330] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: stopping tablet replica
I20260812 07:12:48.830529 16067 raft_consensus.cc:2229] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:48.831002 16067 raft_consensus.cc:2258] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:48.833072 16067 ts_tablet_manager.cc:1905] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:48.840819 16067 ts_tablet_manager.cc:1918] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:48.841058 16067 log.cc:1196] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/74f1c8b67de14a0aae5c2bf8be9ab964
I20260812 07:12:48.841697 16067 ts_tablet_manager.cc:1939] T 74f1c8b67de14a0aae5c2bf8be9ab964 P f2af8fcf4ae442ec8032cb5329a4f132: Deleting consensus metadata
I20260812 07:12:48.843518 15167 catalog_manager.cc:4841] TS f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413): tablet 74f1c8b67de14a0aae5c2bf8be9ab964 (table test-table [id=655bbec0fc764c6398b278c9ae7b4d7b]) successfully deleted
I20260812 07:12:48.844391 16064 tablet_replica.cc:330] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: stopping tablet replica
I20260812 07:12:48.844858 16064 raft_consensus.cc:2229] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:48.845399 16064 raft_consensus.cc:2258] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:48.846217 16066 tablet_replica.cc:330] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: stopping tablet replica
I20260812 07:12:48.846609 16066 raft_consensus.cc:2229] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:48.847214 16066 raft_consensus.cc:2258] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:48.849290 16064 ts_tablet_manager.cc:1905] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 12 07:12:48 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:12:48.853812 16066 ts_tablet_manager.cc:1905] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:48.856608 16064 ts_tablet_manager.cc:1918] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:48.856842 16064 log.cc:1196] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/74f1c8b67de14a0aae5c2bf8be9ab964
I20260812 07:12:48.857409 16064 ts_tablet_manager.cc:1939] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 1fb26d543d1f4d4191403a9819b68151: Deleting consensus metadata
I20260812 07:12:48.857704 16025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:48.830568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46868 (local address 127.21.181.65:40413)
0812 07:12:48.830753 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:48.830759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:48.833656 (+ 2897us) server_negotiation.cc:407] Connection header received
0812 07:12:48.833833 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:48.833839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:48.833919 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:48.834030 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:48.842494 (+ 8464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.843700 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.845944 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:48.846426 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:48.846503 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:48.850824 (+ 4321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:48.850835 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:48.850847 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:48.850881 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:48.854124 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:48.855113 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:48.855122 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:48.855126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:48.855286 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:48.855748 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:48.855753 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:48.855757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:48.856237 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:48.856525 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:48.857134 (+ 609us) server_negotiation.cc:299] Negotiation successful
0812 07:12:48.857345 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:12:48.859265 15167 catalog_manager.cc:4841] TS 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66:34253): tablet 74f1c8b67de14a0aae5c2bf8be9ab964 (table test-table [id=655bbec0fc764c6398b278c9ae7b4d7b]) successfully deleted
I20260812 07:12:48.859381 15490 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:48.861898 16066 ts_tablet_manager.cc:1918] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:48.862085 16066 log.cc:1196] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/74f1c8b67de14a0aae5c2bf8be9ab964
I20260812 07:12:48.862635 16066 ts_tablet_manager.cc:1939] T 74f1c8b67de14a0aae5c2bf8be9ab964 P 8125d793dc11415ebcb7d465e3545ccb: Deleting consensus metadata
I20260812 07:12:48.864281 15167 catalog_manager.cc:4841] TS 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67:33957): tablet 74f1c8b67de14a0aae5c2bf8be9ab964 (table test-table [id=655bbec0fc764c6398b278c9ae7b4d7b]) successfully deleted
I20260812 07:12:48.972256 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 15297
I20260812 07:12:49.001988 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 15509
Aug 12 07:12:49 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:12:49.018867 16022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:49.002771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.21.181.66:34253)
0812 07:12:49.002942 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:49.002946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:49.002966 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:49.003154 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:49.003158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:49.003222 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:49.003301 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:49.004931 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:49.005689 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:49.007448 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:49.007921 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:49.008017 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:49.013058 (+ 5041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:49.013072 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:49.013087 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:49.013124 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:49.015854 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.016938 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.016946 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.016948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.017084 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.017505 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.017508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.017510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.017892 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:49.018124 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:49.018442 (+ 318us) server_negotiation.cc:299] Negotiation successful
0812 07:12:49.018602 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:12:49.019558 15738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:49.106529 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 15509
I20260812 07:12:49.131263 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 15757
Aug 12 07:12:49 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:12:49.150185 16027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:49.132040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.21.181.67:33957)
0812 07:12:49.132247 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:49.132253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:49.132335 (+ 82us) server_negotiation.cc:407] Connection header received
0812 07:12:49.132543 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:49.132547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:49.132616 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:49.132706 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:49.133984 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:49.134792 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:49.136152 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:49.136674 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:49.136790 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:49.144330 (+ 7540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:49.144346 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:49.144365 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:49.144411 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:49.147303 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.148002 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.148010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.148012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.148216 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:49.148608 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:49.148613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:49.148617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:49.148988 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:49.149287 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:49.149639 (+ 352us) server_negotiation.cc:299] Negotiation successful
0812 07:12:49.149842 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260812 07:12:49.150832 15993 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:49.291108 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 15757
I20260812 07:12:49.315758 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 15123
I20260812 07:12:49.316833 15238 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:49.422093 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 15123
I20260812 07:12:49.450171 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33479
--webserver_interface=127.21.181.126
--webserver_port=35691
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:49.777107 16082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:49.777407 16082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:49.777542 16082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:49.784408 16082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:49.784511 16082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:49.784586 16082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:49.784667 16082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:49.784725 16082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:49.793893 16082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:33479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=35691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:49.795023 16082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:49.796244 16082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:49.819535 16098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:49.819487 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:49.819569 16093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:49.820163 16082 server_base.cc:1034] running on GCE node
I20260812 07:12:49.821031 16082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:49.822546 16082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:49.823753 16082 hybrid_clock.cc:648] HybridClock initialized: now 1786518769823704 us; error 35 us; skew 500 ppm
I20260812 07:12:49.824182 16082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:49 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518769, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:49.827482 16082 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:49.829049 16082 webserver.cc:460] Webserver started at http://127.21.181.126:35691/ using document root <none> and password file <none>
I20260812 07:12:49.829741 16082 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:49.829870 16082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:49.835618 16082 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:12:49.838550 16117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:49.839867 16082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:12:49.840041 16082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "b06169228d9a4eb7886295daa107969c"
format_stamp: "Formatted at 2026-08-12 07:12:46 on dist-test-slave-lgr5"
server_key: "9e7d06e7aa2d368b197b2cf107ed1e11"
server_key_iv: "935f8177cba5fdd254054184b9c9f853"
server_key_version: "encryptionkey@0"
I20260812 07:12:49.841084 16082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:49.854784 16082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:49.858511 16082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:49.858758 16082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:49.876225 16082 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:33479
I20260812 07:12:49.876262 16214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:33479 every 8 connection(s)
I20260812 07:12:49.877647 16082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:49.884333 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 16082
I20260812 07:12:49.884548 16215 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:12:49.885344 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:40413
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:33479
--webserver_port=34423
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:49.888924 16215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap starting.
I20260812 07:12:49.909754 16215 log.cc:824] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:49.921694 16215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:12:49.922174 16215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap complete.
I20260812 07:12:49.930927 16215 raft_consensus.cc:348] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:49.931715 16215 raft_consensus.cc:729] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06169228d9a4eb7886295daa107969c, State: Initialized, Role: FOLLOWER
I20260812 07:12:49.932329 16215 consensus_queue.cc:260] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:49.932562 16215 raft_consensus.cc:388] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:49.932679 16215 raft_consensus.cc:482] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:49.932839 16215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:12:49.936533 16215 raft_consensus.cc:504] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:49.937005 16215 leader_election.cc:302] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b06169228d9a4eb7886295daa107969c; no voters:
I20260812 07:12:49.937559 16215 leader_election.cc:288] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:12:49.937726 16222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:12:49.938846 16222 raft_consensus.cc:686] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 LEADER]: Becoming Leader. State: Replica: b06169228d9a4eb7886295daa107969c, State: Running, Role: LEADER
I20260812 07:12:49.939536 16222 consensus_queue.cc:237] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:49.939842 16215 sys_catalog.cc:564] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:49.942154 16226 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b06169228d9a4eb7886295daa107969c. Latest consensus state: current_term: 2 leader_uuid: "b06169228d9a4eb7886295daa107969c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } } }
I20260812 07:12:49.942526 16226 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:49.943099 16224 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b06169228d9a4eb7886295daa107969c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } } }
I20260812 07:12:49.943382 16224 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:49.953573 16229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:49.961911 16229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:49.963415 16229 catalog_manager.cc:1259] Loaded cluster ID: ea5e08c63b3e4caeb3ce23f3bdff762b
I20260812 07:12:49.963567 16229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:12:49.966856 16229 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260812 07:12:49.986384 16229 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260812 07:12:50.012436 16229 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:12:50.014418 16229 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Loaded TSK: 0
I20260812 07:12:50.016131 16229 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:12:50.267894 16219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:50.268187 16219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:50.268337 16219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:50.275452 16219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:50.275549 16219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:50.275686 16219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:12:50.285157 16219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:40413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=34423
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:50.286366 16219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:50.287638 16219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:50.302078 16264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:50.302273 16262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:50.302510 16268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:50.302922 16219 server_base.cc:1034] running on GCE node
I20260812 07:12:50.303483 16219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:50.304409 16219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:50.305603 16219 hybrid_clock.cc:648] HybridClock initialized: now 1786518770305548 us; error 52 us; skew 500 ppm
I20260812 07:12:50.305902 16219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:50 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518770, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:50.312606 16219 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:12:50.314023 16219 webserver.cc:460] Webserver started at http://127.21.181.65:34423/ using document root <none> and password file <none>
I20260812 07:12:50.314659 16219 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:50.314778 16219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:50.320245 16219 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260812 07:12:50.323065 16280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.324549 16219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:50.324712 16219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "70c73d88af35ef6dcf04e1082a0db883"
server_key_iv: "24a2c42cb89d6188f45b93f15024d9f0"
server_key_version: "encryptionkey@0"
I20260812 07:12:50.325683 16219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:50.346742 16219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:50.350909 16219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:50.351151 16219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:50.352183 16219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:50.353516 16219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:50.353621 16219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.353729 16219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:50.353797 16219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.386286 16219 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:40413
I20260812 07:12:50.386334 16475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:40413 every 8 connection(s)
I20260812 07:12:50.387725 16219 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:50.394244 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 16219
I20260812 07:12:50.395009 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:34253
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:33479
--webserver_port=37343
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:12:50 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518770, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:50.410656 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:50.390399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:37721 (local address 127.21.181.126:33479)
0812 07:12:50.391054 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:50.391070 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:50.391503 (+ 433us) server_negotiation.cc:407] Connection header received
0812 07:12:50.393554 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:50.393566 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:50.393880 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:50.394164 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:50.396059 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.397287 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.399249 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.399866 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.399996 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:50.403779 (+ 3783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:50.403786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:50.403799 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:50.403833 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:50.406782 (+ 2949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:50.407462 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:50.407469 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:50.407474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:50.407676 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:50.408161 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:50.408165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:50.408167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:50.408604 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:50.408751 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:50.409514 (+ 763us) server_negotiation.cc:299] Negotiation successful
0812 07:12:50.409799 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":248,"threads_started":1}
I20260812 07:12:50.410946 16481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:50.390556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:37721)
0812 07:12:50.391330 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:50.391374 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:50.393031 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:50.394389 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:50.394399 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:50.395105 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:50.395810 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:50.395835 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.397453 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.397459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:50.399129 (+ 1670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:50.399145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.400119 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.400128 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:50.400383 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:50.401041 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:50.401074 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:50.403535 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:50.406912 (+ 3377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:50.406920 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:50.406934 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:50.407334 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:50.407785 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:50.407790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:50.407793 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:50.408025 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:50.408739 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:50.408746 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:50.409105 (+ 359us) client_negotiation.cc:769] Sending connection context
0812 07:12:50.409365 (+ 260us) client_negotiation.cc:240] Negotiation successful
0812 07:12:50.409818 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":334,"threads_started":1}
I20260812 07:12:50.413954 16476 heartbeater.cc:347] Connected to a master server at 127.21.181.126:33479
I20260812 07:12:50.414222 16476 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:50.414903 16476 heartbeater.cc:511] Master 127.21.181.126:33479 requested a full tablet report, sending...
I20260812 07:12:50.417194 16148 ts_manager.cc:194] Registered new tserver with Master: f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413)
I20260812 07:12:50.419111 16148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:37721
W20260812 07:12:50.716367 16483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:50.716610 16483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:50.716707 16483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:50.723520 16483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:50.723609 16483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:50.723721 16483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:12:50.732825 16483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:34253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=37343
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:50.733930 16483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:50.735129 16483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:50.747681 16497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:50.748112 16500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:50.747706 16489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:50.749653 16483 server_base.cc:1034] running on GCE node
I20260812 07:12:50.750342 16483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:50.751169 16483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:50.752379 16483 hybrid_clock.cc:648] HybridClock initialized: now 1786518770752328 us; error 41 us; skew 500 ppm
I20260812 07:12:50.752669 16483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:50 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518770, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:50.755666 16483 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:12:50.756992 16483 webserver.cc:460] Webserver started at http://127.21.181.66:37343/ using document root <none> and password file <none>
I20260812 07:12:50.757604 16483 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:50.757711 16483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:50.762646 16483 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 07:12:50.765125 16513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.766460 16483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:12:50.766615 16483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1fb26d543d1f4d4191403a9819b68151"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "cb7a516c1b806ddfd6ec6f17b19a976d"
server_key_iv: "6b45d2277779abbf9d14dc87b8cfbf72"
server_key_version: "encryptionkey@0"
I20260812 07:12:50.767547 16483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:50.809590 16483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:50.813709 16483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:50.813967 16483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:50.814917 16483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:50.816370 16483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:50.816470 16483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.816568 16483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:50.816637 16483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:50.848989 16483 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:34253
I20260812 07:12:50.849071 16716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:34253 every 8 connection(s)
I20260812 07:12:50.850378 16483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:50.854132 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 16483
I20260812 07:12:50.854754 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:33957
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:33479
--webserver_port=36495
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:12:50 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518770, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:50.869343 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:50.852674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:35629 (local address 127.21.181.126:33479)
0812 07:12:50.852879 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:50.852885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:50.853995 (+ 1110us) server_negotiation.cc:407] Connection header received
0812 07:12:50.855913 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:50.855919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:50.855998 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:50.856105 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:50.857539 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.858247 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.859747 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.860204 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.860316 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:50.863677 (+ 3361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:50.863683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:50.863687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:50.863725 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:50.866288 (+ 2563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:50.867079 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:50.867083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:50.867085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:50.867203 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:50.867617 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:50.867620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:50.867621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:50.867953 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:50.868141 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:50.868924 (+ 783us) server_negotiation.cc:299] Negotiation successful
0812 07:12:50.869102 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:12:50.870569 16721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:50.853154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:35629)
0812 07:12:50.853841 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:50.853877 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:50.855636 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:50.856353 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:50.856361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:50.856824 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:50.857372 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:50.857390 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.858403 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.858407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:50.859599 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:50.859612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:50.860429 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:50.860438 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:50.860648 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:50.861248 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:50.861270 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:50.863456 (+ 2186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:50.866425 (+ 2969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:50.866433 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:50.866446 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:50.866978 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:50.867299 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:50.867304 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:50.867308 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:50.867532 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:50.868132 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:50.868140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:50.868547 (+ 407us) client_negotiation.cc:769] Sending connection context
0812 07:12:50.868862 (+ 315us) client_negotiation.cc:240] Negotiation successful
0812 07:12:50.869325 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":262,"threads_started":1}
I20260812 07:12:50.871980 16717 heartbeater.cc:347] Connected to a master server at 127.21.181.126:33479
I20260812 07:12:50.872294 16717 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:50.872995 16717 heartbeater.cc:511] Master 127.21.181.126:33479 requested a full tablet report, sending...
I20260812 07:12:50.874511 16148 ts_manager.cc:194] Registered new tserver with Master: 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66:34253)
I20260812 07:12:50.875576 16148 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:35629
W20260812 07:12:51.174435 16724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:51.174678 16724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:51.174779 16724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:51.181607 16724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:51.181699 16724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:51.181810 16724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:12:51.190740 16724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:33957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=36495
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:51.191807 16724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:51.193029 16724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:51.205111 16732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:51.205154 16735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:51.205180 16730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:51.205861 16724 server_base.cc:1034] running on GCE node
I20260812 07:12:51.206401 16724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:51.207283 16724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:51.208509 16724 hybrid_clock.cc:648] HybridClock initialized: now 1786518771208465 us; error 33 us; skew 500 ppm
I20260812 07:12:51.208817 16724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:51 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518771, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:51.211716 16724 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:12:51.213124 16724 webserver.cc:460] Webserver started at http://127.21.181.67:36495/ using document root <none> and password file <none>
I20260812 07:12:51.213806 16724 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:51.213933 16724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:51.218904 16724 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 07:12:51.221455 16745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:51.222647 16724 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:12:51.222788 16724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "8125d793dc11415ebcb7d465e3545ccb"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-lgr5"
server_key: "64448239d02f1ca1a3c87ec209a44129"
server_key_iv: "965558ddfb393687e95b08e388a5d147"
server_key_version: "encryptionkey@0"
I20260812 07:12:51.223696 16724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:51.240697 16724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:51.244328 16724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:51.244556 16724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:51.245437 16724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:51.246631 16724 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:51.246726 16724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:51.246824 16724 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:51.246894 16724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:51.276050 16724 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:33957
I20260812 07:12:51.276147 16950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:33957 every 8 connection(s)
I20260812 07:12:51.277546 16724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:51.283121 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 16724
Aug 12 07:12:51 dist-test-slave-lgr5 krb5kdc[15098](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518771, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:12:51.296990 16480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.279914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:42009 (local address 127.21.181.126:33479)
0812 07:12:51.280154 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.280160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.281238 (+ 1078us) server_negotiation.cc:407] Connection header received
0812 07:12:51.282955 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.282960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.283039 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.283128 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:51.284857 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.285617 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.287107 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.287542 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.287650 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.291037 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:51.291042 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:51.291045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:51.291076 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:51.293940 (+ 2864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.294756 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.294760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.294762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.294895 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.295323 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.295328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.295331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.295657 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:51.295805 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.296581 (+ 776us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.296748 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76,"spinlock_wait_cycles":1408}
I20260812 07:12:51.298043 16953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.280388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:42009)
0812 07:12:51.281093 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.281131 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:51.282695 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:51.283368 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:51.283375 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:51.283867 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:12:51.284686 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.284707 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.285774 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.285777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.286957 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.286970 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.287753 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.287761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.288006 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.288640 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:12:51.288662 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:12:51.290836 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:12:51.294081 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:51.294090 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:51.294103 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:51.294645 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:51.295009 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:12:51.295012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:12:51.295015 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:12:51.295204 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:12:51.295780 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:12:51.295785 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:12:51.296142 (+ 357us) client_negotiation.cc:769] Sending connection context
0812 07:12:51.296440 (+ 298us) client_negotiation.cc:240] Negotiation successful
0812 07:12:51.296868 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":260,"threads_started":1}
I20260812 07:12:51.298156 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.284001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33508 (local address 127.21.181.126:33479)
0812 07:12:51.284753 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.284758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.284771 (+ 13us) server_negotiation.cc:407] Connection header received
0812 07:12:51.284815 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.284818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.284869 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.284968 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:51.286447 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.287186 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.288562 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.288976 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.289084 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.291746 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:51.291753 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:51.291758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:51.291806 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:51.294745 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.295451 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.295455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.295457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.295570 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.295971 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.295976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.295979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.296377 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:51.296615 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.297578 (+ 963us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.297783 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":506,"threads_started":1}
I20260812 07:12:51.299348 16951 heartbeater.cc:347] Connected to a master server at 127.21.181.126:33479
I20260812 07:12:51.299650 16951 heartbeater.cc:464] Registering TS with master...
I20260812 07:12:51.300411 16951 heartbeater.cc:511] Master 127.21.181.126:33479 requested a full tablet report, sending...
I20260812 07:12:51.301753 16147 ts_manager.cc:194] Registered new tserver with Master: 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67:33957)
I20260812 07:12:51.302597 16147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:42009
I20260812 07:12:51.312495 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:12:51.318312 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.313328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33510 (local address 127.21.181.126:33479)
0812 07:12:51.313461 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.313467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.313895 (+ 428us) server_negotiation.cc:407] Connection header received
0812 07:12:51.314048 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.314051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.314165 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.314277 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:51.314620 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.315267 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.316643 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.317041 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.317130 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.317545 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:51.317948 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:51.318050 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.318087 (+ 37us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.318142 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260812 07:12:51.319085 16147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33510:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:12:51.325912 16147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:12:51.346048 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.336999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:33957 (local address 127.0.0.1:59434)
0812 07:12:51.337752 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.337791 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:51.337939 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:51.338938 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:51.338947 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:51.338982 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:51.339446 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.339461 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.341403 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.341407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.343603 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.343618 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.345034 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.345041 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.345295 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.345338 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 07:12:51.345494 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 07:12:51.345702 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":349,"threads_started":1}
I20260812 07:12:51.346393 16970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.337276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59434 (local address 127.21.181.67:33957)
0812 07:12:51.338024 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.338031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.338064 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:12:51.338162 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.338169 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.338400 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.338606 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:51.339673 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.341257 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.343753 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.344829 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.344946 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.345181 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.345913 (+ 732us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.346083 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":417,"threads_started":1}
I20260812 07:12:51.348639 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.339305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.21.181.65:40413)
0812 07:12:51.340813 (+ 1508us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.340822 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.340848 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:12:51.340939 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.340946 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.341163 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.341336 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:51.342072 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.343505 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.345853 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.347220 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.347359 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.347542 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.348186 (+ 644us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.348357 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1367,"thread_start_us":1254,"threads_started":1}
I20260812 07:12:51.348659 16973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.338104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:34253 (local address 127.0.0.1:48946)
0812 07:12:51.338687 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.338705 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:51.338825 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:51.340760 (+ 1935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:51.340764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:51.340789 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:51.341111 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.341122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.343063 (+ 1941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.343069 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.345251 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.345266 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.347913 (+ 2647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.347920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.348111 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.348152 (+ 41us) client_negotiation.cc:769] Sending connection context
0812 07:12:51.348240 (+ 88us) client_negotiation.cc:240] Negotiation successful
0812 07:12:51.348356 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":313,"threads_started":1}
I20260812 07:12:51.348374 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.338762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59788)
0812 07:12:51.339428 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.339444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:51.339598 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:51.341347 (+ 1749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:51.341351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:51.341376 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:51.341882 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.341893 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.343656 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.343662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.345714 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.345725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.347456 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.347463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.347644 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.347676 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:12:51.347771 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:12:51.347922 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":6,"thread_start_us":679,"threads_started":1}
I20260812 07:12:51.348932 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.338400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48946 (local address 127.21.181.66:34253)
0812 07:12:51.340138 (+ 1738us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.340146 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.340180 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:12:51.340275 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.340282 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.340509 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.341329 (+ 820us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:51.341595 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.342893 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.345416 (+ 2523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.347685 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.347805 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.348064 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.348459 (+ 395us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.348624 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533,"thread_start_us":1905,"threads_started":1}
I20260812 07:12:51.350262 16829 tablet_service.cc:1467] Processing CreateTablet for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a (DEFAULT_TABLE table=test-table [id=cf55bc28799f4a13b4bab1a08625210f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:51.351835 16829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:51.351951 16599 tablet_service.cc:1467] Processing CreateTablet for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a (DEFAULT_TABLE table=test-table [id=cf55bc28799f4a13b4bab1a08625210f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:51.353037 16599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:51.352973 16361 tablet_service.cc:1467] Processing CreateTablet for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a (DEFAULT_TABLE table=test-table [id=cf55bc28799f4a13b4bab1a08625210f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:12:51.354280 16361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:12:51.364384 16979 tablet_bootstrap.cc:492] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Bootstrap starting.
I20260812 07:12:51.366843 16981 tablet_bootstrap.cc:492] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Bootstrap starting.
I20260812 07:12:51.367619 16979 tablet_bootstrap.cc:654] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:51.369462 16979 log.cc:824] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:51.370275 16981 tablet_bootstrap.cc:654] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:51.371547 16981 log.cc:824] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:51.372602 16979 tablet_bootstrap.cc:492] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: No bootstrap required, opened a new log
I20260812 07:12:51.372848 16979 ts_tablet_manager.cc:1397] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260812 07:12:51.373926 16981 tablet_bootstrap.cc:492] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: No bootstrap required, opened a new log
I20260812 07:12:51.374154 16981 ts_tablet_manager.cc:1397] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260812 07:12:51.370814 16986 tablet_bootstrap.cc:492] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Bootstrap starting.
I20260812 07:12:51.379685 16979 raft_consensus.cc:348] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.380003 16979 raft_consensus.cc:374] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:51.380070 16986 tablet_bootstrap.cc:654] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Neither blocks nor log segments found. Creating new log.
I20260812 07:12:51.380120 16979 raft_consensus.cc:729] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fb26d543d1f4d4191403a9819b68151, State: Initialized, Role: FOLLOWER
I20260812 07:12:51.380650 16979 consensus_queue.cc:260] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.381250 16986 log.cc:824] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:51.381567 16717 heartbeater.cc:503] Master 127.21.181.126:33479 was elected leader, sending a full tablet report...
I20260812 07:12:51.382866 16979 ts_tablet_manager.cc:1428] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260812 07:12:51.382805 16981 raft_consensus.cc:348] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.383260 16981 raft_consensus.cc:374] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:51.383358 16981 raft_consensus.cc:729] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8125d793dc11415ebcb7d465e3545ccb, State: Initialized, Role: FOLLOWER
I20260812 07:12:51.383845 16981 consensus_queue.cc:260] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.384205 16986 tablet_bootstrap.cc:492] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: No bootstrap required, opened a new log
I20260812 07:12:51.384524 16986 ts_tablet_manager.cc:1397] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.002s
I20260812 07:12:51.384853 16951 heartbeater.cc:503] Master 127.21.181.126:33479 was elected leader, sending a full tablet report...
I20260812 07:12:51.385342 16981 ts_tablet_manager.cc:1428] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Time spent starting tablet: real 0.011s user 0.010s sys 0.002s
I20260812 07:12:51.394497 16986 raft_consensus.cc:348] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.394804 16986 raft_consensus.cc:374] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:12:51.394898 16986 raft_consensus.cc:729] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2af8fcf4ae442ec8032cb5329a4f132, State: Initialized, Role: FOLLOWER
I20260812 07:12:51.395421 16986 consensus_queue.cc:260] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.396389 16476 heartbeater.cc:503] Master 127.21.181.126:33479 was elected leader, sending a full tablet report...
I20260812 07:12:51.396806 16986 ts_tablet_manager.cc:1428] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Time spent starting tablet: real 0.012s user 0.006s sys 0.004s
I20260812 07:12:51.514233 16989 raft_consensus.cc:482] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:12:51.514551 16989 raft_consensus.cc:504] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.516197 16989 leader_election.cc:288] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67:33957), f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413)
I20260812 07:12:51.522521 16970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.516562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:40473 (local address 127.21.181.67:33957)
0812 07:12:51.516742 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.516746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.517400 (+ 654us) server_negotiation.cc:407] Connection header received
0812 07:12:51.517645 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.517650 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.517730 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.517833 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:51.518417 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.519136 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.520890 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.521666 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.521811 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.521885 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.522157 (+ 272us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.522279 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:12:51.522568 16995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.516552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:33957 (local address 127.21.181.66:40473)
0812 07:12:51.517265 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.517285 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:51.517475 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:51.517871 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:51.517876 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:51.517908 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:51.518267 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.518278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.519264 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.519268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.520731 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.520744 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.521845 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.521850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.522001 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.522025 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:12:51.522105 (+ 80us) client_negotiation.cc:240] Negotiation successful
0812 07:12:51.522226 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":392,"threads_started":1}
I20260812 07:12:51.523464 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.517679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:60751 (local address 127.21.181.65:40413)
0812 07:12:51.517910 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.517916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.518202 (+ 286us) server_negotiation.cc:407] Connection header received
0812 07:12:51.518412 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.518416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.518478 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.518560 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:12:51.519149 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.519873 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.521654 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.522451 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.522624 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.522704 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.523067 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.523199 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:12:51.523514 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.517450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.21.181.66:60751)
0812 07:12:51.518096 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.518117 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:12:51.518263 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:12:51.518589 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:12:51.518594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:12:51.518621 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:12:51.518981 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.518991 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.520027 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.520031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.521481 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:12:51.521496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.522682 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.522689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:12:51.522889 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.522928 (+ 39us) client_negotiation.cc:769] Sending connection context
0812 07:12:51.523019 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:12:51.523141 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":461,"threads_started":1}
I20260812 07:12:51.523530 16869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d01bc1991604fd0a3d4de3f2aeebf3a" candidate_uuid: "1fb26d543d1f4d4191403a9819b68151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8125d793dc11415ebcb7d465e3545ccb" is_pre_election: true
I20260812 07:12:51.524173 16869 raft_consensus.cc:2454] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26d543d1f4d4191403a9819b68151 in term 0.
I20260812 07:12:51.524600 16403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d01bc1991604fd0a3d4de3f2aeebf3a" candidate_uuid: "1fb26d543d1f4d4191403a9819b68151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" is_pre_election: true
I20260812 07:12:51.524809 16518 leader_election.cc:302] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26d543d1f4d4191403a9819b68151, 8125d793dc11415ebcb7d465e3545ccb; no voters:
I20260812 07:12:51.525152 16989 raft_consensus.cc:2781] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:12:51.525281 16989 raft_consensus.cc:482] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:12:51.525188 16403 raft_consensus.cc:2454] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1fb26d543d1f4d4191403a9819b68151 in term 0.
I20260812 07:12:51.525382 16989 raft_consensus.cc:3037] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:51.527807 16989 raft_consensus.cc:504] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.528354 16989 leader_election.cc:288] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [CANDIDATE]: Term 1 election: Requested vote from peers 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67:33957), f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413)
I20260812 07:12:51.528863 16403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d01bc1991604fd0a3d4de3f2aeebf3a" candidate_uuid: "1fb26d543d1f4d4191403a9819b68151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
I20260812 07:12:51.528859 16869 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d01bc1991604fd0a3d4de3f2aeebf3a" candidate_uuid: "1fb26d543d1f4d4191403a9819b68151" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8125d793dc11415ebcb7d465e3545ccb"
I20260812 07:12:51.529034 16403 raft_consensus.cc:3037] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 0 FOLLOWER]: Advancing to term 1
W20260812 07:12:51.529110 16952 tablet.cc:2369] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:12:51.529110 16869 raft_consensus.cc:3037] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:12:51.531203 16403 raft_consensus.cc:2454] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26d543d1f4d4191403a9819b68151 in term 1.
I20260812 07:12:51.531530 16518 leader_election.cc:302] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fb26d543d1f4d4191403a9819b68151, f2af8fcf4ae442ec8032cb5329a4f132; no voters:
I20260812 07:12:51.531761 16989 raft_consensus.cc:2781] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:12:51.531771 16869 raft_consensus.cc:2454] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1fb26d543d1f4d4191403a9819b68151 in term 1.
I20260812 07:12:51.532228 16989 raft_consensus.cc:686] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 1 LEADER]: Becoming Leader. State: Replica: 1fb26d543d1f4d4191403a9819b68151, State: Running, Role: LEADER
I20260812 07:12:51.532739 16989 consensus_queue.cc:237] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } }
I20260812 07:12:51.536929 16148 catalog_manager.cc:5495] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "1fb26d543d1f4d4191403a9819b68151" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fb26d543d1f4d4191403a9819b68151" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 34253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 } health_report { overall_health: UNKNOWN } } }
I20260812 07:12:51.583086 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.578041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48956 (local address 127.21.181.66:34253)
0812 07:12:51.578250 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.578256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.578274 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:12:51.578364 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.578368 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.578439 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.578567 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:12:51.578921 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.579632 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.581056 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.581457 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.581567 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.582054 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:12:51.582519 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:12:51.582659 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.582760 (+ 101us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.582840 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:12:51.590662 16403 raft_consensus.cc:1261] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Refusing update from remote peer 1fb26d543d1f4d4191403a9819b68151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:51.590664 16869 raft_consensus.cc:1261] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Refusing update from remote peer 1fb26d543d1f4d4191403a9819b68151: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:12:51.591521 16999 consensus_queue.cc:1035] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2af8fcf4ae442ec8032cb5329a4f132" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 40413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:51.591895 16989 consensus_queue.cc:1035] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8125d793dc11415ebcb7d465e3545ccb" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:12:51.602130 17005 mvcc.cc:205] Tried to move back new op lower bound from 7317580888426352640 to 7317580888201883648. Current Snapshot: MvccSnapshot[applied={T|T < 7317580888426352640}]
I20260812 07:12:51.614671 16147 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:51.614892 16147 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33510:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:12:51.618084 16147 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Sending DeleteTablet for 3 replicas of tablet 8d01bc1991604fd0a3d4de3f2aeebf3a
I20260812 07:12:51.619043 16829 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:51 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:59434
I20260812 07:12:51.619195 16599 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:51 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:48946
I20260812 07:12:51.619421 16361 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:12:51 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:59788
I20260812 07:12:51.619897 17025 tablet_replica.cc:330] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: stopping tablet replica
I20260812 07:12:51.619906 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 16219
I20260812 07:12:51.620191 17027 tablet_replica.cc:330] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: stopping tablet replica
I20260812 07:12:51.620301 17025 raft_consensus.cc:2229] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:51.620704 17027 raft_consensus.cc:2229] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:12:51.620780 17025 raft_consensus.cc:2258] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:51.621403 17027 raft_consensus.cc:2258] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:51.623422 17027 ts_tablet_manager.cc:1905] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:51.626089 17025 ts_tablet_manager.cc:1905] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:51.627897 17030 tablet_replica.cc:330] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: stopping tablet replica
I20260812 07:12:51.628293 17030 raft_consensus.cc:2229] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:12:51.628729 17030 raft_consensus.cc:2258] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:12:51.630944 17027 ts_tablet_manager.cc:1918] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:51.631114 17027 log.cc:1196] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/8d01bc1991604fd0a3d4de3f2aeebf3a
I20260812 07:12:51.631672 17027 ts_tablet_manager.cc:1939] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 1fb26d543d1f4d4191403a9819b68151: Deleting consensus metadata
I20260812 07:12:51.633383 16124 catalog_manager.cc:4841] TS 1fb26d543d1f4d4191403a9819b68151 (127.21.181.66:34253): tablet 8d01bc1991604fd0a3d4de3f2aeebf3a (table test-table [id=cf55bc28799f4a13b4bab1a08625210f]) successfully deleted
I20260812 07:12:51.635192 17030 ts_tablet_manager.cc:1905] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:51.637858 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.621473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.21.181.65:40413)
0812 07:12:51.621652 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.621658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.621677 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:12:51.621729 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.621734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.621814 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.621912 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:51.623691 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.624822 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.626655 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.627338 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.627464 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.630802 (+ 3338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:51.630815 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:51.630831 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:51.630873 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:51.634551 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.635409 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.635418 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.635422 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.635591 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.636017 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.636022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.636026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.636616 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:51.636915 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.637262 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.637483 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:12:51.637837 17025 ts_tablet_manager.cc:1918] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:51.637987 17025 log.cc:1196] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/8d01bc1991604fd0a3d4de3f2aeebf3a
I20260812 07:12:51.638468 17025 ts_tablet_manager.cc:1939] T 8d01bc1991604fd0a3d4de3f2aeebf3a P 8125d793dc11415ebcb7d465e3545ccb: Deleting consensus metadata
I20260812 07:12:51.638831 16470 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:51.640149 16124 catalog_manager.cc:4841] TS 8125d793dc11415ebcb7d465e3545ccb (127.21.181.67:33957): tablet 8d01bc1991604fd0a3d4de3f2aeebf3a (table test-table [id=cf55bc28799f4a13b4bab1a08625210f]) successfully deleted
I20260812 07:12:51.726495 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 16219
I20260812 07:12:51.747308 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 16483
W20260812 07:12:51.747219 16124 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.181.65:40413 (error 108)
I20260812 07:12:51.749828 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.749307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59810)
0812 07:12:51.749512 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.749636 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260812 07:12:51.750097 16124 catalog_manager.cc:4584] TS f2af8fcf4ae442ec8032cb5329a4f132 (127.21.181.65:40413): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d01bc1991604fd0a3d4de3f2aeebf3a: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
I20260812 07:12:51.767753 16974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.748476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48958 (local address 127.21.181.66:34253)
0812 07:12:51.748673 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.748679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.748699 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:12:51.748775 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.748781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.748861 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.748976 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:51.750680 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.751855 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.753798 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.754529 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.754663 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.761092 (+ 6429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:51.761109 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:51.761127 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:51.761175 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:51.764973 (+ 3798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.765623 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.765633 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.765637 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.765816 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.766145 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.766151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.766154 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.766614 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:51.766962 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.767127 (+ 165us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.767349 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260812 07:12:51.768507 16708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:51.803612 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.803176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59820)
0812 07:12:51.803369 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.803462 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:12:51.804594 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.804253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59834)
0812 07:12:51.804398 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.804476 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:12:51.865044 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.864600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59842)
0812 07:12:51.864801 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.864893 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:12:51.866066 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.865729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59854)
0812 07:12:51.865881 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.865952 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260812 07:12:51.880117 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 16483
I20260812 07:12:51.907039 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 16724
I20260812 07:12:51.919035 16970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.907697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.21.181.67:33957)
0812 07:12:51.907850 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:12:51.907854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:12:51.907909 (+ 55us) server_negotiation.cc:407] Connection header received
0812 07:12:51.908060 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:12:51.908064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:12:51.908169 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:12:51.908249 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:12:51.909483 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.910181 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.911392 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:12:51.911765 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:12:51.911860 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:12:51.914383 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:12:51.914397 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:12:51.914412 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:12:51.914446 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:12:51.916853 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.917494 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.917500 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.917503 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.917639 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:12:51.917912 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:12:51.917915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:12:51.917917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:12:51.918247 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:12:51.918466 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0812 07:12:51.918643 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:12:51.918791 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:12:51.919631 16944 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:51.947129 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.946619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59858)
0812 07:12:51.946828 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.946931 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:12:51.948138 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:51.947716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59866)
0812 07:12:51.947901 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:51.947981 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 07:12:52.010613 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 16724
I20260812 07:12:52.033797 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 16082
I20260812 07:12:52.034916 16205 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:12:52.146414 16976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:52.145705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:40413 (local address 127.0.0.1:59878)
0812 07:12:52.146014 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:52.146124 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:40413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
I20260812 07:12:52.146698 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 16082
I20260812 07:12:52.175130 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33479
--webserver_interface=127.21.181.126
--webserver_port=35691
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:33479
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:12:52.499953 17041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:52.500334 17041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:52.500511 17041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:52.507635 17041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:12:52.507748 17041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:52.507851 17041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:52.507932 17041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:12:52.508001 17041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:12:52.517627 17041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:33479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:33479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=35691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:52.518806 17041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:52.520073 17041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:52.531121 17052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:52.531208 17053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:52.531153 17057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:52.532239 17041 server_base.cc:1034] running on GCE node
I20260812 07:12:52.532955 17041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:52.534199 17041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:52.535406 17041 hybrid_clock.cc:648] HybridClock initialized: now 1786518772535354 us; error 37 us; skew 500 ppm
I20260812 07:12:52.535712 17041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:52 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518772, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:52.538791 17041 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:12:52.540246 17041 webserver.cc:460] Webserver started at http://127.21.181.126:35691/ using document root <none> and password file <none>
I20260812 07:12:52.540903 17041 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:52.541041 17041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:52.546247 17041 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260812 07:12:52.549014 17070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:52.550212 17041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260812 07:12:52.550383 17041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "b06169228d9a4eb7886295daa107969c"
format_stamp: "Formatted at 2026-08-12 07:12:46 on dist-test-slave-lgr5"
server_key: "9e7d06e7aa2d368b197b2cf107ed1e11"
server_key_iv: "935f8177cba5fdd254054184b9c9f853"
server_key_version: "encryptionkey@0"
I20260812 07:12:52.551358 17041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:52.570983 17041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:52.574608 17041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:52.574865 17041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:52.592929 17041 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:33479
I20260812 07:12:52.592934 17156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:33479 every 8 connection(s)
I20260812 07:12:52.594360 17041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:12:52.597954 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 17041
I20260812 07:12:52.598898 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:40413
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:33479
--webserver_port=34423
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:52.601946 17160 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:12:52.606149 17160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap starting.
I20260812 07:12:52.626317 17160 log.cc:824] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Log is configured to *not* fsync() on all Append() calls
I20260812 07:12:52.641826 17160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:12:52.642302 17160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c: Bootstrap complete.
I20260812 07:12:52.650771 17160 raft_consensus.cc:348] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:52.651666 17160 raft_consensus.cc:729] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b06169228d9a4eb7886295daa107969c, State: Initialized, Role: FOLLOWER
I20260812 07:12:52.652271 17160 consensus_queue.cc:260] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:52.652482 17160 raft_consensus.cc:388] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:12:52.652590 17160 raft_consensus.cc:482] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:12:52.652745 17160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:12:52.656502 17160 raft_consensus.cc:504] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:52.656949 17160 leader_election.cc:302] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b06169228d9a4eb7886295daa107969c; no voters:
I20260812 07:12:52.657526 17160 leader_election.cc:288] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:12:52.657651 17166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:12:52.658695 17166 raft_consensus.cc:686] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [term 3 LEADER]: Becoming Leader. State: Replica: b06169228d9a4eb7886295daa107969c, State: Running, Role: LEADER
I20260812 07:12:52.659264 17166 consensus_queue.cc:237] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } }
I20260812 07:12:52.659493 17160 sys_catalog.cc:564] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:12:52.661780 17169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: SysCatalogTable state changed. Reason: New leader b06169228d9a4eb7886295daa107969c. Latest consensus state: current_term: 3 leader_uuid: "b06169228d9a4eb7886295daa107969c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } } }
I20260812 07:12:52.661811 17168 sys_catalog.cc:455] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b06169228d9a4eb7886295daa107969c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b06169228d9a4eb7886295daa107969c" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 33479 } } }
I20260812 07:12:52.662195 17168 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:52.662194 17169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b06169228d9a4eb7886295daa107969c [sys.catalog]: This master's current role is: LEADER
I20260812 07:12:52.665500 17177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:12:52.674734 17177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:12:52.676301 17177 catalog_manager.cc:1259] Loaded cluster ID: ea5e08c63b3e4caeb3ce23f3bdff762b
I20260812 07:12:52.676446 17177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260812 07:12:52.678519 17177 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0e889b59ad google::LogMessage::Fail() at ??:0
@ 0x7f0e889b888a google::LogMessage::SendToLog() at ??:0
@ 0x7f0e889b54de google::LogMessage::Flush() at ??:0
@ 0x7f0e889b6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0e9e3e8512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0e9e3e5976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0e9e46d610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0e9e46d439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0e8b601345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0e8b60491c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0e8b604799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0e8b5dfeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0e8e8b96db start_thread at ??:0
@ 0x7f0e8784f61f clone at ??:0
W20260812 07:12:53.000592 17164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:53.000840 17164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:53.000932 17164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:53.007953 17164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:53.008049 17164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:53.008199 17164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:12:53.017553 17164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:40413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=34423
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:53.018631 17164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:53.019810 17164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:53.031909 17226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.031958 17228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.032135 17230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:53.032842 17164 server_base.cc:1034] running on GCE node
I20260812 07:12:53.033360 17164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:53.034152 17164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:53.035342 17164 hybrid_clock.cc:648] HybridClock initialized: now 1786518773035296 us; error 41 us; skew 500 ppm
I20260812 07:12:53.035637 17164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:53 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518773, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:53.038594 17164 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:12:53.039933 17164 webserver.cc:460] Webserver started at http://127.21.181.65:34423/ using document root <none> and password file <none>
I20260812 07:12:53.040596 17164 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:53.040715 17164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:53.045666 17164 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:12:53.050220 17241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.051412 17164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260812 07:12:53.051573 17164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "f2af8fcf4ae442ec8032cb5329a4f132"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "70c73d88af35ef6dcf04e1082a0db883"
server_key_iv: "24a2c42cb89d6188f45b93f15024d9f0"
server_key_version: "encryptionkey@0"
I20260812 07:12:53.052529 17164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:53.083250 17164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:53.086802 17164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:53.087033 17164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:53.087960 17164 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:53.089620 17262 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 07:12:53.094581 17262 ts_tablet_manager.cc:1852] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 07:12:53.095525 17262 ts_tablet_manager.cc:1905] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:12:53.102239 17262 ts_tablet_manager.cc:1918] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:12:53.102416 17262 log.cc:1196] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/8d01bc1991604fd0a3d4de3f2aeebf3a
I20260812 07:12:53.102834 17262 ts_tablet_manager.cc:1939] T 8d01bc1991604fd0a3d4de3f2aeebf3a P f2af8fcf4ae442ec8032cb5329a4f132: Deleting consensus metadata
I20260812 07:12:53.103183 17164 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 07:12:53.103296 17164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260812 07:12:53.103399 17164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:53.103472 17164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.135357 17164 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:40413
I20260812 07:12:53.135411 17450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:40413 every 8 connection(s)
I20260812 07:12:53.136715 17164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:12:53.137719 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 17164
I20260812 07:12:53.138510 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:34253
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:33479
--webserver_port=37343
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:53.142115 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:53.139945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:35529)
0812 07:12:53.140807 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:53.141084 (+ 277us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":312,"threads_started":1}
W20260812 07:12:53.142769 17452 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:33479 (0 consecutive failures): Network error: Failed to ping master at 127.21.181.126:33479: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
I20260812 07:12:53.143599 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:53.143257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:43635)
0812 07:12:53.143405 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:53.143482 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:12:53.144532 17458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:53.144164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:52757)
0812 07:12:53.144297 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:53.144397 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:12:53.144778 17452 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:12:53.457620 17461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:53.457862 17461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:53.458006 17461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:53.464875 17461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:53.464979 17461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:53.465131 17461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:12:53.474228 17461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:34253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=37343
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:53.475446 17461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:53.476799 17461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:53.488965 17475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.489074 17468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.489055 17472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:53.490060 17461 server_base.cc:1034] running on GCE node
I20260812 07:12:53.490603 17461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:53.491442 17461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:53.492626 17461 hybrid_clock.cc:648] HybridClock initialized: now 1786518773492575 us; error 46 us; skew 500 ppm
I20260812 07:12:53.492904 17461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:53 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518773, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:53.495823 17461 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:12:53.497172 17461 webserver.cc:460] Webserver started at http://127.21.181.66:37343/ using document root <none> and password file <none>
I20260812 07:12:53.497787 17461 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:53.497900 17461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:53.502821 17461 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260812 07:12:53.505352 17489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.506600 17461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:12:53.506748 17461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1fb26d543d1f4d4191403a9819b68151"
format_stamp: "Formatted at 2026-08-12 07:12:47 on dist-test-slave-lgr5"
server_key: "cb7a516c1b806ddfd6ec6f17b19a976d"
server_key_iv: "6b45d2277779abbf9d14dc87b8cfbf72"
server_key_version: "encryptionkey@0"
I20260812 07:12:53.507691 17461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:53.533973 17461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:53.537429 17461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:53.537671 17461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:53.538556 17461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:53.539795 17461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:53.539889 17461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.539994 17461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:53.540063 17461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.570024 17461 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:34253
I20260812 07:12:53.570098 17682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:34253 every 8 connection(s)
I20260812 07:12:53.571336 17461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:12:53.574501 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 17461
I20260812 07:12:53.575238 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:33957
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:33479
--webserver_port=36495
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:46431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:12:53.575861 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:53.573989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:54819)
0812 07:12:53.574769 (+ 780us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:53.574983 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":309,"threads_started":1}
W20260812 07:12:53.576476 17683 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:33479 (0 consecutive failures): Network error: Failed to ping master at 127.21.181.126:33479: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
I20260812 07:12:53.577239 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:53.576930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:35059)
0812 07:12:53.577072 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:53.577140 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:12:53.578024 17685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:53.577719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:37711)
0812 07:12:53.577849 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:53.577920 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:12:53.578243 17683 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:12:53.894646 17688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:12:53.894886 17688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:12:53.894974 17688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:12:53.901880 17688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:12:53.901971 17688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:12:53.902089 17688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:12:53.911346 17688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:46431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:33957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=36495
--tserver_master_addrs=127.21.181.126:33479
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:12:53.912496 17688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:12:53.913720 17688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:12:53.925848 17695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.925913 17699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:12:53.925887 17694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:12:53.926723 17688 server_base.cc:1034] running on GCE node
I20260812 07:12:53.927250 17688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:12:53.928126 17688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:12:53.929318 17688 hybrid_clock.cc:648] HybridClock initialized: now 1786518773929273 us; error 40 us; skew 500 ppm
I20260812 07:12:53.929615 17688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:12:53 dist-test-slave-lgr5 krb5kdc[15098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518773, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:12:53.932502 17688 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:12:53.933736 17688 webserver.cc:460] Webserver started at http://127.21.181.67:36495/ using document root <none> and password file <none>
I20260812 07:12:53.934348 17688 fs_manager.cc:359] Metadata directory not provided
I20260812 07:12:53.934464 17688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:12:53.939414 17688 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:12:53.941957 17714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.943151 17688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:12:53.943302 17688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "8125d793dc11415ebcb7d465e3545ccb"
format_stamp: "Formatted at 2026-08-12 07:12:48 on dist-test-slave-lgr5"
server_key: "64448239d02f1ca1a3c87ec209a44129"
server_key_iv: "965558ddfb393687e95b08e388a5d147"
server_key_version: "encryptionkey@0"
I20260812 07:12:53.944263 17688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:12:53.960157 17688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:12:53.963857 17688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:12:53.964103 17688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:12:53.964965 17688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:12:53.966142 17688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:12:53.966245 17688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.966346 17688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:12:53.966419 17688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:12:53.997540 17688 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:33957
I20260812 07:12:53.997603 17932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:33957 every 8 connection(s)
I20260812 07:12:53.998929 17688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:12:54.000366 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 17688
I20260812 07:12:54.003508 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.001802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:60527)
0812 07:12:54.002493 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:54.002682 (+ 189us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":283,"threads_started":1}
W20260812 07:12:54.004154 17935 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:33479 (0 consecutive failures): Network error: Failed to ping master at 127.21.181.126:33479: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
I20260812 07:12:54.004899 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.004597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:42155)
0812 07:12:54.004737 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:54.004799 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260812 07:12:54.005669 17940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.005365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:57513)
0812 07:12:54.005498 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:54.005568 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:12:54.005885 17935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:12:54.145967 17942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.145239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:37717)
0812 07:12:54.145654 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:54.145762 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":194,"threads_started":1}
I20260812 07:12:54.579433 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:54.578679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:47663)
0812 07:12:54.579110 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:54.579222 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":204,"threads_started":1}
I20260812 07:12:55.007146 17945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:55.006355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:42815)
0812 07:12:55.006795 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:55.006919 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":217,"threads_started":1}
I20260812 07:12:55.147313 17946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:55.146701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:33423)
0812 07:12:55.147093 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:55.147171 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":205,"threads_started":1}
I20260812 07:12:55.580883 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:55.580236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:38663)
0812 07:12:55.580652 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:55.580728 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":200,"threads_started":1}
I20260812 07:12:56.008647 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:56.007914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:42821)
0812 07:12:56.008404 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:56.008484 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":264,"threads_started":1}
I20260812 07:12:56.148782 17960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:56.148072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:54331)
0812 07:12:56.148517 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:56.148598 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":216,"threads_started":1}
I20260812 07:12:56.582398 17962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:56.581679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:45729)
0812 07:12:56.582153 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:56.582229 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":250,"threads_started":1}
I20260812 07:12:57.010067 17964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:57.009403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:37587)
0812 07:12:57.009825 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:57.009904 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":218,"threads_started":1}
I20260812 07:12:57.150254 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:57.149539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:50949)
0812 07:12:57.149966 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:57.150047 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":214,"threads_started":1}
I20260812 07:12:57.583822 17970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:57.583139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:60083)
0812 07:12:57.583565 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:57.583640 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":209,"threads_started":1}
I20260812 07:12:58.011610 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:58.010809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:46955)
0812 07:12:58.011307 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:58.011389 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":246,"threads_started":1}
I20260812 07:12:58.151787 17972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:58.151087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:33531)
0812 07:12:58.151542 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:58.151629 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":234,"threads_started":1}
I20260812 07:12:58.585222 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:58.584546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:50551)
0812 07:12:58.584987 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:58.585063 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":209,"threads_started":1}
I20260812 07:12:59.013090 17975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:59.012425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:33807)
0812 07:12:59.012855 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:59.012931 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":215,"threads_started":1}
I20260812 07:12:59.153203 17976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:59.152517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:45453)
0812 07:12:59.152968 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:59.153049 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":218,"threads_started":1}
I20260812 07:12:59.586628 17977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:12:59.585965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:43401)
0812 07:12:59.586382 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0812 07:12:59.586471 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":210,"threads_started":1}
I20260812 07:13:00.014580 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:00.013888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:57835)
0812 07:13:00.014335 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:00.014419 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
I20260812 07:13:00.154938 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:00.154049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:56575)
0812 07:13:00.154618 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:00.154717 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":274,"threads_started":1}
I20260812 07:13:00.588285 17981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:00.587442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:47035)
0812 07:13:00.587912 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:00.587991 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":201,"threads_started":1}
I20260812 07:13:01.016345 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:01.015434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:36031)
0812 07:13:01.016009 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:01.016131 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":250,"threads_started":1}
I20260812 07:13:01.156522 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:01.155780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:47893)
0812 07:13:01.156265 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:01.156350 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":276,"threads_started":1}
I20260812 07:13:01.589787 17986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:01.589076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:52241)
0812 07:13:01.589540 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:01.589627 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":241,"threads_started":1}
I20260812 07:13:02.017833 17989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:02.017096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:50155)
0812 07:13:02.017577 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:02.017665 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":253,"threads_started":1}
I20260812 07:13:02.157971 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:02.157257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:50765)
0812 07:13:02.157709 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:02.157808 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":234,"threads_started":1}
I20260812 07:13:02.591308 17994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:02.590568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:55551)
0812 07:13:02.591055 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:02.591139 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":224,"threads_started":1}
I20260812 07:13:03.019531 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:03.018717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:46019)
0812 07:13:03.019258 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:03.019350 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":260,"threads_started":1}
I20260812 07:13:03.159583 17999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:03.158861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:41213)
0812 07:13:03.159322 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:03.159414 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260812 07:13:03.593009 18003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:03.592266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:44613)
0812 07:13:03.592751 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:03.592844 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":256,"threads_started":1}
I20260812 07:13:04.021039 18004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:04.020348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:51965)
0812 07:13:04.020799 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:04.020879 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":239,"threads_started":1}
I20260812 07:13:04.161147 18005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:04.160412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:60179)
0812 07:13:04.160871 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:04.160953 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":254,"threads_started":1}
I20260812 07:13:04.594489 18006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:04.593816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:38737)
0812 07:13:04.594225 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:04.594318 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":198,"threads_started":1}
I20260812 07:13:05.022519 18008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:05.021806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:46077)
0812 07:13:05.022230 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:05.022320 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":204,"threads_started":1}
I20260812 07:13:05.162634 18009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:05.161974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:51585)
0812 07:13:05.162396 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:05.162473 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":235,"threads_started":1}
I20260812 07:13:05.596002 18010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:05.595335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:45527)
0812 07:13:05.595770 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:05.595849 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":229,"threads_started":1}
I20260812 07:13:06.024034 18013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:06.023315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:40113)
0812 07:13:06.023781 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:06.023866 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":229,"threads_started":1}
I20260812 07:13:06.164196 18014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:06.163409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:44119)
0812 07:13:06.163858 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:06.163956 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
I20260812 07:13:06.597564 18016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:06.596819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:35037)
0812 07:13:06.597293 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:06.597381 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":252,"threads_started":1}
I20260812 07:13:07.025588 18019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:07.024861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:55743)
0812 07:13:07.025321 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:07.025409 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":227,"threads_started":1}
I20260812 07:13:07.165774 18020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:07.165058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:59281)
0812 07:13:07.165511 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:07.165591 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":227,"threads_started":1}
I20260812 07:13:07.599061 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:07.598378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:54903)
0812 07:13:07.598801 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:07.598879 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":221,"threads_started":1}
I20260812 07:13:08.027288 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.026343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:41455)
0812 07:13:08.026888 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.027053 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":254,"threads_started":1}
I20260812 07:13:08.167479 18026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.166726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.65:41155)
0812 07:13:08.167171 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.167266 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":230,"threads_started":1}
I20260812 07:13:08.600813 18028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:08.600044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.66:51785)
0812 07:13:08.600524 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:08.600635 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":230,"threads_started":1}
I20260812 07:13:09.006191 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 17164
I20260812 07:13:09.027808 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.007154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.21.181.65:40413)
0812 07:13:09.007743 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.007762 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.007837 (+ 75us) server_negotiation.cc:407] Connection header received
0812 07:13:09.009953 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.009964 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.010420 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.010726 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.012030 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.013298 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.014420 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.015085 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.015214 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.022472 (+ 7258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.022480 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.022494 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.022534 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.025095 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.025674 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.025680 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.025686 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.025843 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.026131 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.026134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.026136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.026611 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.026829 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.027122 (+ 293us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.027482 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":246,"threads_started":1}
I20260812 07:13:09.029126 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.028346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:33479 (local address 127.21.181.67:48269)
0812 07:13:09.028823 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:09.028942 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:33479: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":231,"threads_started":1}
I20260812 07:13:09.029314 17444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:09.111105 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 17164
I20260812 07:13:09.139276 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 17461
I20260812 07:13:09.156016 18037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.140275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.21.181.66:34253)
0812 07:13:09.140892 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.140906 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.140942 (+ 36us) server_negotiation.cc:407] Connection header received
0812 07:13:09.142490 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.142509 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.142919 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.143236 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.144742 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.145958 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.147270 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.147984 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.148202 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.150562 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.150569 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.150582 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.150618 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.153274 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.153901 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.153907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.153913 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.154079 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.154367 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.154370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.154372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.154855 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.155089 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.155418 (+ 329us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.155738 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":326,"threads_started":1}
I20260812 07:13:09.157321 17675 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:09.237298 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 17461
I20260812 07:13:09.261358 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 17688
I20260812 07:13:09.277563 18043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:09.262265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.21.181.67:33957)
0812 07:13:09.262798 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:09.262811 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:09.262854 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:13:09.264561 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:09.264572 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:09.265001 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:09.265277 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:09.266493 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.267664 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.269042 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:09.269668 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:09.269796 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:09.272141 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:09.272150 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:09.272164 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:09.272202 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:09.274688 (+ 2486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.275293 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.275298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.275304 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.275462 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:09.275740 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:09.275743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:09.275745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:09.276344 (+ 599us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:09.276556 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:09.276876 (+ 320us) server_negotiation.cc:299] Negotiation successful
0812 07:13:09.277250 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":223,"threads_started":1}
I20260812 07:13:09.278820 17925 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:09.380447 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 17688
2026-08-12T07:13:09Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24420 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:13:09 dist-test-slave-lgr5 krb5kdc[18050](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:13:09 dist-test-slave-lgr5 krb5kdc[18050](info): set up 2 sockets
Aug 12 07:13:09 dist-test-slave-lgr5 krb5kdc[18050](info): commencing operation
krb5kdc: starting...
W20260812 07:13:10.639695 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.004s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:13:10 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:13:10.880986 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38495
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:38495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:11.200398 18076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:11.200671 18076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:11.200767 18076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:11.207592 18076 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:13:11.207671 18076 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:13:11.207715 18076 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:13:11.207765 18076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:11.207816 18076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:11.207881 18076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:11.207924 18076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:11.207981 18076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:11.217128 18076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:38495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:11.218186 18076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:11.219342 18076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:11.230257 18087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:11.230264 18094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:11.230214 18099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:11.230932 18076 server_base.cc:1034] running on GCE node
I20260812 07:13:11.231663 18076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:11.232950 18076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:11.234163 18076 hybrid_clock.cc:648] HybridClock initialized: now 1786518791234099 us; error 35 us; skew 500 ppm
I20260812 07:13:11.234464 18076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:11 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:11.237517 18076 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:13:11.238817 18076 webserver.cc:460] Webserver started at http://127.21.181.126:45773/ using document root <none> and password file <none>
I20260812 07:13:11.239432 18076 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:11.239542 18076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:11.239817 18076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:11.242609 18076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "19a26a82f6ba49d48da1737da46a74ba"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "d6d1a87c694996831578ae5b05eaf91b"
server_key_iv: "486792be26dc17030f0c16fa14607e1f"
server_key_version: "encryptionkey@0"
I20260812 07:13:11.243322 18076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "19a26a82f6ba49d48da1737da46a74ba"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "d6d1a87c694996831578ae5b05eaf91b"
server_key_iv: "486792be26dc17030f0c16fa14607e1f"
server_key_version: "encryptionkey@0"
I20260812 07:13:11.248050 18076 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260812 07:13:11.251200 18111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:11.252391 18076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:13:11.252552 18076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "19a26a82f6ba49d48da1737da46a74ba"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "d6d1a87c694996831578ae5b05eaf91b"
server_key_iv: "486792be26dc17030f0c16fa14607e1f"
server_key_version: "encryptionkey@0"
I20260812 07:13:11.252736 18076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:11.266786 18076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:11.270219 18076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:11.270439 18076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:11.286616 18076 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:38495
I20260812 07:13:11.286623 18208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:38495 every 8 connection(s)
I20260812 07:13:11.287791 18076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:11.291250 18209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:11.294382 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 18076
I20260812 07:13:11.294615 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:13:11.295487 22229 external_mini_cluster.cc:1428] Setting key fcfb82564363bca93f5284712fc0d331
I20260812 07:13:11.303862 18209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap starting.
I20260812 07:13:11.307315 18209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:11.308614 18209 log.cc:824] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:13:11 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:11.311244 18209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: No bootstrap required, opened a new log
I20260812 07:13:11.318651 18209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:11.318835 18215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:11.297540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60048 (local address 127.21.181.126:38495)
0812 07:13:11.298351 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:11.298362 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:11.298405 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:13:11.300101 (+ 1696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:11.300135 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:11.300538 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:11.300866 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:11.302598 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:11.303962 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:11.305780 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:11.306459 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:11.306592 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:11.311181 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:11.311189 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:11.311203 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:11.311251 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:11.314534 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:11.315277 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:11.315283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:11.315289 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:11.315482 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:11.315915 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:11.315919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:11.315921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:11.316557 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:11.316872 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:11.317372 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 07:13:11.317791 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":384,"threads_started":1}
I20260812 07:13:11.318971 18209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:11.319062 18209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a26a82f6ba49d48da1737da46a74ba, State: Initialized, Role: FOLLOWER
I20260812 07:13:11.319594 18209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:11.319808 18209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:11.319941 18209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:11.320139 18209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:11.322793 18209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:11.323215 18209 leader_election.cc:302] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a26a82f6ba49d48da1737da46a74ba; no voters:
I20260812 07:13:11.323704 18209 leader_election.cc:288] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:13:11.323851 18218 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:11.324553 18218 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 LEADER]: Becoming Leader. State: Replica: 19a26a82f6ba49d48da1737da46a74ba, State: Running, Role: LEADER
I20260812 07:13:11.325060 18218 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:11.325584 18209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:11.327603 18223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a26a82f6ba49d48da1737da46a74ba. Latest consensus state: current_term: 1 leader_uuid: "19a26a82f6ba49d48da1737da46a74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } } }
I20260812 07:13:11.327961 18223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:11.328279 18222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19a26a82f6ba49d48da1737da46a74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } } }
I20260812 07:13:11.329903 18222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:11.332996 18228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:11.338536 18228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:11.346163 18228 catalog_manager.cc:1347] Generated new cluster ID: 09b9e69760534fd6853fb8ece1a66b96
I20260812 07:13:11.346259 18228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:11.362898 18228 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:13:11.364710 18228 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:11.374400 18228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Generated new TSK 0
I20260812 07:13:11.375137 18228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:13:11.445873 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38495
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:11.772464 18254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:11.772737 18254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:11.772882 18254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:11.779824 18254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:11.779930 18254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:11.780107 18254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:13:11.789194 18254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:11.790333 18254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:11.791841 18254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:11.803777 18264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:11.803810 18270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:11.803859 18265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:11.804751 18254 server_base.cc:1034] running on GCE node
I20260812 07:13:11.805244 18254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:11.806042 18254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:11.807224 18254 hybrid_clock.cc:648] HybridClock initialized: now 1786518791807174 us; error 42 us; skew 500 ppm
I20260812 07:13:11.807471 18254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:11 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:11.810511 18254 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:13:11.812683 18254 webserver.cc:460] Webserver started at http://127.21.181.65:35387/ using document root <none> and password file <none>
I20260812 07:13:11.813511 18254 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:11.813647 18254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:11.814000 18254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:11.816893 18254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "e3de0038337543bea06d86bb1a0ef5dd"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "43f49456d6f7f2e466d20f96191441b9"
server_key_iv: "c2ff5464d5239754886de75f2b5c4e69"
server_key_version: "encryptionkey@0"
I20260812 07:13:11.817592 18254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "e3de0038337543bea06d86bb1a0ef5dd"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "43f49456d6f7f2e466d20f96191441b9"
server_key_iv: "c2ff5464d5239754886de75f2b5c4e69"
server_key_version: "encryptionkey@0"
I20260812 07:13:11.822561 18254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260812 07:13:11.825817 18283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:11.827139 18254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:13:11.827287 18254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "e3de0038337543bea06d86bb1a0ef5dd"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "43f49456d6f7f2e466d20f96191441b9"
server_key_iv: "c2ff5464d5239754886de75f2b5c4e69"
server_key_version: "encryptionkey@0"
I20260812 07:13:11.827454 18254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:11.848814 18254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:11.852850 18254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:11.853106 18254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:11.853997 18254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:11.855280 18254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:11.855362 18254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:11.855481 18254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:11.855542 18254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:11.885797 18254 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:43987
I20260812 07:13:11.885842 18469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:43987 every 8 connection(s)
I20260812 07:13:11.887040 18254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:13:11.893060 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 18254
I20260812 07:13:11.893263 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:13:11.893873 22229 external_mini_cluster.cc:1428] Setting key 69debe7cfcddd8ce4cf825bc333e6b93
Aug 12 07:13:11 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518791, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:11.905745 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:11.889418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:56221 (local address 127.21.181.126:38495)
0812 07:13:11.889813 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:11.889817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:11.890726 (+ 909us) server_negotiation.cc:407] Connection header received
0812 07:13:11.892573 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:11.892579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:11.892682 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:11.892779 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:11.894318 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:11.895155 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:11.896840 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:11.897252 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:11.897362 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:11.900591 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:11.900596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:11.900599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:11.900632 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:11.902902 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:11.903578 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:11.903582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:11.903585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:11.903708 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:11.904119 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:11.904124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:11.904127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:11.904377 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:11.904513 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:11.905173 (+ 660us) server_negotiation.cc:299] Negotiation successful
0812 07:13:11.905397 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":200,"threads_started":1}
I20260812 07:13:11.906555 18475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:11.889875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:56221)
0812 07:13:11.890573 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:11.890611 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:11.892305 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:11.893023 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:11.893030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:11.893579 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:11.894146 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:11.894171 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:11.895312 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:11.895316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:11.896701 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:11.896714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:11.897464 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:11.897472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:11.897700 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:11.898335 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:11.898356 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:11.900384 (+ 2028us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:11.903036 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:11.903044 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:11.903059 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:11.903449 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:11.903809 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:11.903814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:11.903817 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:11.903990 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:11.904509 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:11.904517 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:11.904805 (+ 288us) client_negotiation.cc:769] Sending connection context
0812 07:13:11.905027 (+ 222us) client_negotiation.cc:240] Negotiation successful
0812 07:13:11.905502 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":256,"threads_started":1}
I20260812 07:13:11.907886 18472 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38495
I20260812 07:13:11.908195 18472 heartbeater.cc:464] Registering TS with master...
I20260812 07:13:11.908844 18472 heartbeater.cc:511] Master 127.21.181.126:38495 requested a full tablet report, sending...
I20260812 07:13:11.910770 18141 ts_manager.cc:194] Registered new tserver with Master: e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987)
I20260812 07:13:11.912197 18141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:56221
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:13:11.950990 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38495
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:12.267691 18478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:12.267957 18478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:12.268106 18478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:12.275105 18478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:12.275197 18478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:12.275339 18478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:13:12.284389 18478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:12.285503 18478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:12.286803 18478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:12.299198 18489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.299230 18492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.299518 18488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:12.299957 18478 server_base.cc:1034] running on GCE node
I20260812 07:13:12.300644 18478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:12.301637 18478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:12.302850 18478 hybrid_clock.cc:648] HybridClock initialized: now 1786518792302803 us; error 38 us; skew 500 ppm
I20260812 07:13:12.303133 18478 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:12 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:12.306095 18478 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:13:12.307384 18478 webserver.cc:460] Webserver started at http://127.21.181.66:37309/ using document root <none> and password file <none>
I20260812 07:13:12.308000 18478 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:12.308132 18478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:12.308444 18478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:12.311177 18478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "7dd47abc8b7249f2959cfa0f04cf164a"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "018187236c8f12cc36640fc2ee5d15a7"
server_key_iv: "07b9663cc3b955c763a8030357b9bd9a"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.311878 18478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "7dd47abc8b7249f2959cfa0f04cf164a"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "018187236c8f12cc36640fc2ee5d15a7"
server_key_iv: "07b9663cc3b955c763a8030357b9bd9a"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.316598 18478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:13:12.319746 18501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.321159 18478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:12.321311 18478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "7dd47abc8b7249f2959cfa0f04cf164a"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "018187236c8f12cc36640fc2ee5d15a7"
server_key_iv: "07b9663cc3b955c763a8030357b9bd9a"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.321525 18478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:12.333341 18478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:12.337002 18478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:12.337251 18478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:12.338086 18478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:12.339267 18478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:12.339344 18478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.339464 18478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:12.339526 18478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.367311 18478 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:35543
I20260812 07:13:12.367353 18704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:35543 every 8 connection(s)
I20260812 07:13:12.368528 18478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:13:12.371538 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 18478
I20260812 07:13:12.371733 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:13:12.372370 22229 external_mini_cluster.cc:1428] Setting key 2babad0946a538e61c4e25e8c4773f8d
Aug 12 07:13:12 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:12.387245 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.370922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:60393 (local address 127.21.181.126:38495)
0812 07:13:12.371110 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.371115 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.372376 (+ 1261us) server_negotiation.cc:407] Connection header received
0812 07:13:12.374143 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.374149 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.374228 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.374314 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.375854 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.376584 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.378056 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.378471 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.378586 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.381934 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.381939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.381942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.381974 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.384456 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.385115 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.385119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.385121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.385248 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.385677 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.385681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.385683 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.385953 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.386078 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.386829 (+ 751us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.386999 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:13:12.388450 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.371423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:60393)
0812 07:13:12.372229 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.372275 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.373884 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.374581 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.374592 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.375166 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.375690 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.375708 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.376727 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.376731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.377919 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.377932 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.378687 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.378695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.378899 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.379474 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.379498 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.381728 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.384592 (+ 2864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.384600 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.384614 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.385002 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.385350 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.385356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.385360 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.385550 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.386073 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.386081 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.386469 (+ 388us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.386768 (+ 299us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.387239 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":333,"threads_started":1}
I20260812 07:13:12.389801 18705 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38495
I20260812 07:13:12.390074 18705 heartbeater.cc:464] Registering TS with master...
I20260812 07:13:12.390712 18705 heartbeater.cc:511] Master 127.21.181.126:38495 requested a full tablet report, sending...
I20260812 07:13:12.392048 18141 ts_manager.cc:194] Registered new tserver with Master: 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66:35543)
I20260812 07:13:12.392843 18141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:60393
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:13:12.431741 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38495
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:12.746867 18713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:12.747119 18713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:12.747246 18713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:12.754164 18713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:12.754266 18713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:12.754408 18713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:13:12.763721 18713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:12.764926 18713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:12.766177 18713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:12.778316 18718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.778239 18722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:12.778282 18720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:12.779206 18713 server_base.cc:1034] running on GCE node
I20260812 07:13:12.779747 18713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:12.780634 18713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:12.781809 18713 hybrid_clock.cc:648] HybridClock initialized: now 1786518792781756 us; error 46 us; skew 500 ppm
I20260812 07:13:12.782078 18713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:12 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:12.784996 18713 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:13:12.786230 18713 webserver.cc:460] Webserver started at http://127.21.181.67:39771/ using document root <none> and password file <none>
I20260812 07:13:12.786836 18713 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:12.786945 18713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:12.787226 18713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:13:12.789937 18713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "107e3873958dcd95fefbcbcc8497d2c0"
server_key_iv: "f92bb717958a6bdc8e956c99c25a41f9"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.790648 18713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "107e3873958dcd95fefbcbcc8497d2c0"
server_key_iv: "f92bb717958a6bdc8e956c99c25a41f9"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.795305 18713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:13:12.798450 18732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.799618 18713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:13:12.799749 18713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "107e3873958dcd95fefbcbcc8497d2c0"
server_key_iv: "f92bb717958a6bdc8e956c99c25a41f9"
server_key_version: "encryptionkey@0"
I20260812 07:13:12.799908 18713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:12.812713 18713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:12.817209 18713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:12.817485 18713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:12.818343 18713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:12.819567 18713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:12.819638 18713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.819743 18713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:12.819795 18713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:12.850045 18713 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38549
I20260812 07:13:12.850118 18942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38549 every 8 connection(s)
I20260812 07:13:12.851136 18713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:13:12.853169 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 18713
I20260812 07:13:12.853372 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:13:12.853965 22229 external_mini_cluster.cc:1428] Setting key 3a541259bfa7e7bfd4d1e1e6aebdf8ea
Aug 12 07:13:12 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518792, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:12.870328 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.853797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:35187 (local address 127.21.181.126:38495)
0812 07:13:12.853964 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.853968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.855338 (+ 1370us) server_negotiation.cc:407] Connection header received
0812 07:13:12.857625 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.857630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.857709 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.857792 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.859236 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.860004 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.861542 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.861956 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.862068 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.865283 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.865288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.865291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.865323 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.867548 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.868216 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.868220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.868222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.868357 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.868776 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.868780 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.868782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.869118 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.869272 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.869856 (+ 584us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.870039 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:13:12.871094 18953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.854429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:35187)
0812 07:13:12.855199 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.855245 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.857341 (+ 2096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.858041 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.858049 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.858536 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:12.859072 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.859090 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.860176 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.860182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.861394 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.861407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.862175 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.862183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.862422 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.862991 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:12.863013 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:12.865094 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:12.867683 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.867689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.867700 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.868073 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.868462 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:12.868467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:12.868471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:12.868656 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:12.869250 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:12.869256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:12.869556 (+ 300us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.869759 (+ 203us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.870119 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":296,"threads_started":1}
I20260812 07:13:12.872282 18949 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38495
I20260812 07:13:12.872536 18949 heartbeater.cc:464] Registering TS with master...
I20260812 07:13:12.873159 18949 heartbeater.cc:511] Master 127.21.181.126:38495 requested a full tablet report, sending...
I20260812 07:13:12.874413 18141 ts_manager.cc:194] Registered new tserver with Master: 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67:38549)
I20260812 07:13:12.875157 18141 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:35187
I20260812 07:13:12.881119 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:13:12.897439 18474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.885494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60060 (local address 127.21.181.126:38495)
0812 07:13:12.885666 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.885672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.886151 (+ 479us) server_negotiation.cc:407] Connection header received
0812 07:13:12.886282 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.886287 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.886362 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.886466 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:12.887819 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.888900 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.890153 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.890691 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.890786 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.892947 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:12.892952 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:12.892955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:12.892984 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:12.895364 (+ 2380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.895918 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.895921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.895923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.896053 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:12.896385 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:12.896387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:12.896389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:12.896676 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:12.896848 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.897067 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.897216 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:13:12.901175 18141 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:13:12.907711 18141 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:13:12.914809 18472 heartbeater.cc:503] Master 127.21.181.126:38495 was elected leader, sending a full tablet report...
I20260812 07:13:12.930647 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.919880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47680 (local address 127.21.181.66:35543)
0812 07:13:12.920384 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.920394 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.921520 (+ 1126us) server_negotiation.cc:407] Connection header received
0812 07:13:12.921807 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.921815 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.922081 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.922239 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:12.923185 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.924818 (+ 1633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.927636 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.929166 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.929280 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.929476 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.930133 (+ 657us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.930320 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":260,"threads_started":1}
I20260812 07:13:12.931492 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.919396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:35543 (local address 127.0.0.1:47680)
0812 07:13:12.921397 (+ 2001us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.921416 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.921629 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.922571 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.922579 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.922609 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:12.922952 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.922967 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.925019 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.925024 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.927445 (+ 2421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.927463 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.929660 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.929666 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.929861 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.929912 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.930881 (+ 969us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.931085 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1880,"thread_start_us":11422,"threads_started":1}
I20260812 07:13:12.935400 18592 tablet_service.cc:1467] Processing CreateTablet for tablet 5abe944be8054c7fa6764e20edd3387d (DEFAULT_TABLE table=test-table [id=6f27f420b513404c9790e9cd249d9b33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:13:12.936939 18592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abe944be8054c7fa6764e20edd3387d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:12.937151 18970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.918804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:56734)
0812 07:13:12.919537 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.919575 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.919764 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.931135 (+ 11371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.931140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.931166 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:12.931523 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.931535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.933443 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.933448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.935088 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.935102 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.936423 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.936430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.936609 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.936646 (+ 37us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.936730 (+ 84us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.936846 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":1366,"threads_started":1}
I20260812 07:13:12.937232 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.920034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.21.181.65:43987)
0812 07:13:12.929723 (+ 9689us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.929731 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.929757 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:13:12.929847 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.929854 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.930066 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.931801 (+ 1735us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:12.932070 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.933324 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.935264 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.936223 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.936349 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.936559 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.936785 (+ 226us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.936940 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9529,"thread_start_us":10306,"threads_started":1}
I20260812 07:13:12.939227 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.918807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38549 (local address 127.0.0.1:59360)
0812 07:13:12.930984 (+ 12177us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:12.931003 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:12.931126 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:12.932309 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:12.932314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:12.932340 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:12.932717 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.932729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.934664 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.934673 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.936699 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:12.936714 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.938463 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.938470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:12.938632 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.938663 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:13:12.938763 (+ 100us) client_negotiation.cc:240] Negotiation successful
0812 07:13:12.938883 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12026,"mutex_wait_us":1967,"thread_start_us":339,"threads_started":1}
I20260812 07:13:12.939320 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:12.919136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59360 (local address 127.21.181.67:38549)
0812 07:13:12.931752 (+ 12616us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:12.931760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:12.931794 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:13:12.931892 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:12.931900 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:12.932122 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:12.932305 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:12.932898 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.934453 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.936913 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:12.938249 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:12.938374 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:12.938583 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:12.938824 (+ 241us) server_negotiation.cc:299] Negotiation successful
0812 07:13:12.938971 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12374,"thread_start_us":12219,"threads_started":1}
I20260812 07:13:12.940362 18367 tablet_service.cc:1467] Processing CreateTablet for tablet 5abe944be8054c7fa6764e20edd3387d (DEFAULT_TABLE table=test-table [id=6f27f420b513404c9790e9cd249d9b33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:13:12.941361 18367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abe944be8054c7fa6764e20edd3387d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:12.942577 18832 tablet_service.cc:1467] Processing CreateTablet for tablet 5abe944be8054c7fa6764e20edd3387d (DEFAULT_TABLE table=test-table [id=6f27f420b513404c9790e9cd249d9b33]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:13:12.943885 18832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5abe944be8054c7fa6764e20edd3387d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:12.951792 18982 tablet_bootstrap.cc:492] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Bootstrap starting.
I20260812 07:13:12.952041 18983 tablet_bootstrap.cc:492] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Bootstrap starting.
I20260812 07:13:12.954715 18982 tablet_bootstrap.cc:654] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:12.955430 18983 tablet_bootstrap.cc:654] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:12.955741 18982 log.cc:824] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:12.956650 18983 log.cc:824] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:12.961338 18986 tablet_bootstrap.cc:492] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Bootstrap starting.
I20260812 07:13:12.962579 18983 tablet_bootstrap.cc:492] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: No bootstrap required, opened a new log
I20260812 07:13:12.962818 18983 ts_tablet_manager.cc:1397] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260812 07:13:12.965005 18986 tablet_bootstrap.cc:654] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:12.965998 18986 log.cc:824] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:12.967576 18982 tablet_bootstrap.cc:492] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: No bootstrap required, opened a new log
I20260812 07:13:12.967839 18982 ts_tablet_manager.cc:1397] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260812 07:13:12.968918 18986 tablet_bootstrap.cc:492] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: No bootstrap required, opened a new log
I20260812 07:13:12.969174 18986 ts_tablet_manager.cc:1397] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:13:12.970504 18983 raft_consensus.cc:348] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:12.970932 18983 raft_consensus.cc:374] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:12.971029 18983 raft_consensus.cc:729] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3de0038337543bea06d86bb1a0ef5dd, State: Initialized, Role: FOLLOWER
I20260812 07:13:12.971642 18983 consensus_queue.cc:260] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:12.973274 18983 ts_tablet_manager.cc:1428] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260812 07:13:12.977952 18982 raft_consensus.cc:348] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:12.978392 18982 raft_consensus.cc:374] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:12.978492 18982 raft_consensus.cc:729] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd47abc8b7249f2959cfa0f04cf164a, State: Initialized, Role: FOLLOWER
I20260812 07:13:12.979081 18982 consensus_queue.cc:260] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:12.979240 18986 raft_consensus.cc:348] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:12.979653 18986 raft_consensus.cc:374] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:12.979749 18986 raft_consensus.cc:729] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94a0fd14d7814ddb84555a5c5b9a1a6b, State: Initialized, Role: FOLLOWER
I20260812 07:13:12.980175 18705 heartbeater.cc:503] Master 127.21.181.126:38495 was elected leader, sending a full tablet report...
I20260812 07:13:12.980348 18986 consensus_queue.cc:260] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:12.980832 18982 ts_tablet_manager.cc:1428] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260812 07:13:12.981541 18949 heartbeater.cc:503] Master 127.21.181.126:38495 was elected leader, sending a full tablet report...
I20260812 07:13:12.981954 18986 ts_tablet_manager.cc:1428] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260812 07:13:13.015305 18994 raft_consensus.cc:482] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:13:13.015571 18994 raft_consensus.cc:504] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:13.017266 18994 leader_election.cc:288] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67:38549), e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987)
I20260812 07:13:13.024190 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.017550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:48215 (local address 127.21.181.67:38549)
0812 07:13:13.017742 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.017748 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.018592 (+ 844us) server_negotiation.cc:407] Connection header received
0812 07:13:13.018880 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.018885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.018961 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.019037 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:13.019770 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.020579 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.022499 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.023254 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.023388 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.023460 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:13.023800 (+ 340us) server_negotiation.cc:299] Negotiation successful
0812 07:13:13.023921 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:13:13.024195 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.017765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:55793 (local address 127.21.181.65:43987)
0812 07:13:13.017963 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.017970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.018849 (+ 879us) server_negotiation.cc:407] Connection header received
0812 07:13:13.019020 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.019024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.019087 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.019173 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:13.019903 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.020624 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.022467 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.023261 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.023406 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.023477 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:13.023815 (+ 338us) server_negotiation.cc:299] Negotiation successful
0812 07:13:13.023933 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:13:13.024469 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.017685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38549 (local address 127.21.181.66:48215)
0812 07:13:13.018476 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:13.018498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:13.018705 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:13.019247 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:13.019251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:13.019278 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:13.019621 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:13.019630 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.020708 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.020713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:13.022351 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:13.022364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.023482 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.023491 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:13.023627 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.023679 (+ 52us) client_negotiation.cc:769] Sending connection context
0812 07:13:13.023751 (+ 72us) client_negotiation.cc:240] Negotiation successful
0812 07:13:13.023867 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"mutex_wait_us":103,"thread_start_us":417,"threads_started":1}
I20260812 07:13:13.024540 19000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.017741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.21.181.66:55793)
0812 07:13:13.018742 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:13.018759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:13.018888 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:13.019281 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:13.019286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:13.019312 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:13.019742 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:13.019754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.020758 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.020765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:13.022318 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:13.022333 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.023530 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.023536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:13.023670 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.023693 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:13:13.023775 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 07:13:13.023893 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":882,"mutex_wait_us":6,"thread_start_us":642,"threads_started":1}
I20260812 07:13:13.025377 18859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe944be8054c7fa6764e20edd3387d" candidate_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" is_pre_election: true
I20260812 07:13:13.025517 18399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe944be8054c7fa6764e20edd3387d" candidate_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3de0038337543bea06d86bb1a0ef5dd" is_pre_election: true
I20260812 07:13:13.025914 18859 raft_consensus.cc:2454] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd47abc8b7249f2959cfa0f04cf164a in term 0.
I20260812 07:13:13.026081 18399 raft_consensus.cc:2454] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7dd47abc8b7249f2959cfa0f04cf164a in term 0.
I20260812 07:13:13.026538 18509 leader_election.cc:302] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd47abc8b7249f2959cfa0f04cf164a, 94a0fd14d7814ddb84555a5c5b9a1a6b; no voters:
I20260812 07:13:13.026949 18994 raft_consensus.cc:2781] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:13:13.027110 18994 raft_consensus.cc:482] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:13:13.027211 18994 raft_consensus.cc:3037] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:13.029526 18994 raft_consensus.cc:504] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:13.030118 18994 leader_election.cc:288] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [CANDIDATE]: Term 1 election: Requested vote from peers 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67:38549), e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987)
I20260812 07:13:13.030411 18859 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe944be8054c7fa6764e20edd3387d" candidate_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b"
I20260812 07:13:13.030619 18859 raft_consensus.cc:3037] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:13.030646 18399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5abe944be8054c7fa6764e20edd3387d" candidate_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3de0038337543bea06d86bb1a0ef5dd"
I20260812 07:13:13.030838 18399 raft_consensus.cc:3037] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:13.032902 18859 raft_consensus.cc:2454] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd47abc8b7249f2959cfa0f04cf164a in term 1.
I20260812 07:13:13.033159 18399 raft_consensus.cc:2454] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7dd47abc8b7249f2959cfa0f04cf164a in term 1.
I20260812 07:13:13.033221 18509 leader_election.cc:302] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7dd47abc8b7249f2959cfa0f04cf164a, 94a0fd14d7814ddb84555a5c5b9a1a6b; no voters:
I20260812 07:13:13.033566 18994 raft_consensus.cc:2781] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:13.034008 18994 raft_consensus.cc:686] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 LEADER]: Becoming Leader. State: Replica: 7dd47abc8b7249f2959cfa0f04cf164a, State: Running, Role: LEADER
I20260812 07:13:13.034564 18994 consensus_queue.cc:237] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:13.038700 18140 catalog_manager.cc:5495] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66). New cstate: current_term: 1 leader_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } health_report { overall_health: HEALTHY } } }
I20260812 07:13:13.053930 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.048889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47688 (local address 127.21.181.66:35543)
0812 07:13:13.049077 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.049083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.049100 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:13:13.049259 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.049263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.049339 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.049456 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:13.049822 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.050505 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.051877 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.052319 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.052423 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.052918 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:13.053358 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:13.053484 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:13.053614 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 07:13:13.053688 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:13:13.060990 18399 raft_consensus.cc:1261] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Refusing update from remote peer 7dd47abc8b7249f2959cfa0f04cf164a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:13.060997 18859 raft_consensus.cc:1261] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Refusing update from remote peer 7dd47abc8b7249f2959cfa0f04cf164a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:13.061832 18994 consensus_queue.cc:1035] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:13.062093 19002 consensus_queue.cc:1035] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [LEADER]: Connected to new peer: Peer: permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:13.072745 19009 mvcc.cc:205] Tried to move back new op lower bound from 7317580976369373184 to 7317580976273395712. Current Snapshot: MvccSnapshot[applied={T|T < 7317580976369373184}]
I20260812 07:13:13.073523 19014 mvcc.cc:205] Tried to move back new op lower bound from 7317580976369373184 to 7317580976273395712. Current Snapshot: MvccSnapshot[applied={T|T < 7317580976369373184}]
I20260812 07:13:13.086166 18140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:13:13.086400 18140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:13:13.089619 18140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Sending DeleteTablet for 3 replicas of tablet 5abe944be8054c7fa6764e20edd3387d
I20260812 07:13:13.090615 18592 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abe944be8054c7fa6764e20edd3387d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:13:13 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:47680
I20260812 07:13:13.090615 18832 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abe944be8054c7fa6764e20edd3387d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:13:13 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:59360
I20260812 07:13:13.090631 18367 tablet_service.cc:1514] Processing DeleteTablet for tablet 5abe944be8054c7fa6764e20edd3387d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:13:13 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:56734
I20260812 07:13:13.091182 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 18254
I20260812 07:13:13.092116 19034 tablet_replica.cc:330] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: stopping tablet replica
I20260812 07:13:13.092507 19034 raft_consensus.cc:2229] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:13:13.092986 19034 raft_consensus.cc:2258] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:13:13.093694 19038 tablet_replica.cc:330] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: stopping tablet replica
I20260812 07:13:13.094051 19038 raft_consensus.cc:2229] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:13:13.094481 19038 raft_consensus.cc:2258] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:13:13.095722 19034 ts_tablet_manager.cc:1905] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:13.096451 19038 ts_tablet_manager.cc:1905] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:13.102208 19034 ts_tablet_manager.cc:1918] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:13.102356 19034 log.cc:1196] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/5abe944be8054c7fa6764e20edd3387d
I20260812 07:13:13.102861 19034 ts_tablet_manager.cc:1939] T 5abe944be8054c7fa6764e20edd3387d P 94a0fd14d7814ddb84555a5c5b9a1a6b: Deleting consensus metadata
I20260812 07:13:13.102907 19038 ts_tablet_manager.cc:1918] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:13.103071 19038 log.cc:1196] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/5abe944be8054c7fa6764e20edd3387d
I20260812 07:13:13.103567 19038 ts_tablet_manager.cc:1939] T 5abe944be8054c7fa6764e20edd3387d P e3de0038337543bea06d86bb1a0ef5dd: Deleting consensus metadata
Aug 12 07:13:13 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:13:13.104974 18121 catalog_manager.cc:4841] TS 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67:38549): tablet 5abe944be8054c7fa6764e20edd3387d (table test-table [id=6f27f420b513404c9790e9cd249d9b33]) successfully deleted
I20260812 07:13:13.105165 18120 catalog_manager.cc:4841] TS e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987): tablet 5abe944be8054c7fa6764e20edd3387d (table test-table [id=6f27f420b513404c9790e9cd249d9b33]) successfully deleted
I20260812 07:13:13.108703 19039 tablet_replica.cc:330] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: stopping tablet replica
I20260812 07:13:13.109127 19039 raft_consensus.cc:2229] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:13:13.109856 19039 raft_consensus.cc:2258] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:13:13.111505 19039 ts_tablet_manager.cc:1905] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:13.112840 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.092408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.21.181.65:43987)
0812 07:13:13.092573 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.092579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.092602 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:13:13.092827 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.092834 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.092913 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.092979 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.095795 (+ 2816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.096999 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.098953 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.099623 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.099722 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.105984 (+ 6262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:13.105999 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:13.106016 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:13.106056 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:13.109755 (+ 3699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:13.110567 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:13.110576 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:13.110580 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:13.110729 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:13.111166 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:13.111171 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:13.111174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:13.111627 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:13.111914 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:13.112286 (+ 372us) server_negotiation.cc:299] Negotiation successful
0812 07:13:13.112490 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260812 07:13:13.113440 18465 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:13.117923 19039 ts_tablet_manager.cc:1918] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:13.118083 19039 log.cc:1196] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/5abe944be8054c7fa6764e20edd3387d
I20260812 07:13:13.118513 19039 ts_tablet_manager.cc:1939] T 5abe944be8054c7fa6764e20edd3387d P 7dd47abc8b7249f2959cfa0f04cf164a: Deleting consensus metadata
I20260812 07:13:13.119761 18113 catalog_manager.cc:4841] TS 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66:35543): tablet 5abe944be8054c7fa6764e20edd3387d (table test-table [id=6f27f420b513404c9790e9cd249d9b33]) successfully deleted
I20260812 07:13:13.243777 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 18254
I20260812 07:13:13.268270 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 18478
Aug 12 07:13:13 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:13:13.283957 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.269077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47696 (local address 127.21.181.66:35543)
0812 07:13:13.269264 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.269270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.269349 (+ 79us) server_negotiation.cc:407] Connection header received
0812 07:13:13.269546 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.269550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.269616 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.269717 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.271031 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.271831 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.273222 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.273665 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.273777 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.278173 (+ 4396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:13.278185 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:13.278199 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:13.278236 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:13.281164 (+ 2928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:13.281852 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:13.281860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:13.281862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:13.282013 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:13.282392 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:13.282396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:13.282398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:13.282796 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:13.283113 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:13.283370 (+ 257us) server_negotiation.cc:299] Negotiation successful
0812 07:13:13.283582 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:13:13.284672 18696 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:13.394115 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 18478
I20260812 07:13:13.416007 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 18713
Aug 12 07:13:13 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:13:13.430464 18971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:13.416746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59372 (local address 127.21.181.67:38549)
0812 07:13:13.416939 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:13.416943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:13.416962 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:13:13.417106 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:13.417110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:13.417173 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:13.417253 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:13.418513 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.419230 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.420616 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:13.421000 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:13.421092 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:13.425065 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:13.425080 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:13.425096 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:13.425135 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:13.427836 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:13.428616 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:13.428623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:13.428626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:13.428772 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:13.429094 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:13.429097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:13.429099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:13.429433 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:13.429660 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:13.430003 (+ 343us) server_negotiation.cc:299] Negotiation successful
0812 07:13:13.430172 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:13:13.431063 18937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:13.516731 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 18713
I20260812 07:13:13.539312 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 18076
I20260812 07:13:13.540403 18203 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:13.647185 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 18076
I20260812 07:13:33.675287 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38495
--webserver_interface=127.21.181.126
--webserver_port=45773
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:38495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:34.008313 19056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:34.008592 19056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:34.008733 19056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:34.015743 19056 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:13:34.015829 19056 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:13:34.015892 19056 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:13:34.015964 19056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:34.016034 19056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:34.016148 19056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:34.016202 19056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:34.016276 19056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:34.025682 19056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:38495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=45773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:34.026872 19056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:34.028230 19056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:34.051263 19064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:34.051288 19065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:34.051223 19072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:34.051620 19056 server_base.cc:1034] running on GCE node
I20260812 07:13:34.052706 19056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:34.054157 19056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:34.055383 19056 hybrid_clock.cc:648] HybridClock initialized: now 1786518814055316 us; error 36 us; skew 500 ppm
I20260812 07:13:34.055747 19056 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:34 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518814, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:34.059168 19056 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:13:34.060909 19056 webserver.cc:460] Webserver started at http://127.21.181.126:45773/ using document root <none> and password file <none>
I20260812 07:13:34.061641 19056 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:34.061764 19056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:34.067454 19056 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:13:34.070257 19083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:34.071492 19056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260812 07:13:34.071646 19056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "19a26a82f6ba49d48da1737da46a74ba"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "d6d1a87c694996831578ae5b05eaf91b"
server_key_iv: "486792be26dc17030f0c16fa14607e1f"
server_key_version: "encryptionkey@0"
I20260812 07:13:34.072678 19056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:34.098112 19056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:34.101887 19056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:34.102154 19056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:34.119421 19056 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:38495
I20260812 07:13:34.119457 19182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:38495 every 8 connection(s)
I20260812 07:13:34.120857 19056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:34.123236 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 19056
I20260812 07:13:34.124027 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:43987
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:38495
--webserver_port=35387
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:34.127995 19183 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:13:34.131234 19183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap starting.
I20260812 07:13:34.155197 19183 log.cc:824] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:34.170742 19183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:13:34.171329 19183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap complete.
I20260812 07:13:34.184433 19183 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:34.185563 19183 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a26a82f6ba49d48da1737da46a74ba, State: Initialized, Role: FOLLOWER
I20260812 07:13:34.186261 19183 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:34.186558 19183 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:34.186684 19183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:34.186859 19183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 1 FOLLOWER]: Advancing to term 2
I20260812 07:13:34.192432 19183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:34.192927 19183 leader_election.cc:302] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a26a82f6ba49d48da1737da46a74ba; no voters:
I20260812 07:13:34.193490 19183 leader_election.cc:288] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260812 07:13:34.193662 19188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Leader election won for term 2
I20260812 07:13:34.194682 19188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 LEADER]: Becoming Leader. State: Replica: 19a26a82f6ba49d48da1737da46a74ba, State: Running, Role: LEADER
I20260812 07:13:34.195240 19188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:34.195605 19183 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:34.197889 19191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a26a82f6ba49d48da1737da46a74ba. Latest consensus state: current_term: 2 leader_uuid: "19a26a82f6ba49d48da1737da46a74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } } }
I20260812 07:13:34.198304 19191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:34.198333 19189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "19a26a82f6ba49d48da1737da46a74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } } }
I20260812 07:13:34.198724 19189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:34.209467 19205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:34.217344 19205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:34.218676 19205 catalog_manager.cc:1259] Loaded cluster ID: 09b9e69760534fd6853fb8ece1a66b96
I20260812 07:13:34.218768 19205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:34.222671 19205 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:34.224140 19205 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Loaded TSK: 0
I20260812 07:13:34.253430 19205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Generated new TSK 1
I20260812 07:13:34.253803 19205 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:13:34.478217 19186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:34.478473 19186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:34.478606 19186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:34.485724 19186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:34.485839 19186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:34.486017 19186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:13:34.496026 19186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:43987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=35387
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:34.497359 19186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:34.498622 19186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:34.513196 19227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:34.513154 19229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:34.516781 19226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:34.517208 19186 server_base.cc:1034] running on GCE node
I20260812 07:13:34.517762 19186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:34.518700 19186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:34.519912 19186 hybrid_clock.cc:648] HybridClock initialized: now 1786518814519862 us; error 42 us; skew 500 ppm
I20260812 07:13:34.520243 19186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:34 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518814, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:34.526077 19186 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:13:34.527525 19186 webserver.cc:460] Webserver started at http://127.21.181.65:35387/ using document root <none> and password file <none>
I20260812 07:13:34.528213 19186 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:34.528354 19186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:34.533454 19186 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:13:34.536235 19244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:34.537679 19186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:13:34.537823 19186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "e3de0038337543bea06d86bb1a0ef5dd"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "43f49456d6f7f2e466d20f96191441b9"
server_key_iv: "c2ff5464d5239754886de75f2b5c4e69"
server_key_version: "encryptionkey@0"
I20260812 07:13:34.538749 19186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:34.566002 19186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:34.570019 19186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:34.570267 19186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:34.571165 19186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:34.572532 19186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:34.572621 19186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:34.572698 19186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:34.572757 19186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:34.602772 19186 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:43987
I20260812 07:13:34.602838 19451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:43987 every 8 connection(s)
I20260812 07:13:34.604143 19186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:13:34.607862 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 19186
I20260812 07:13:34.608532 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:35543
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:38495
--webserver_port=37309
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:13:34 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518814, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:34.627274 19459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:34.607040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:41763)
0812 07:13:34.607865 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:34.607908 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:34.609695 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:34.611236 (+ 1541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:34.611243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:34.611732 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:34.612301 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:34.612321 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:34.614370 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:34.614373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:34.615594 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:34.615607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:34.616894 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:34.616899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:34.617088 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:34.617691 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:34.617713 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:34.619938 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:34.623683 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:34.623689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:34.623699 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:34.624134 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:34.624584 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:34.624587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:34.624590 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:34.624754 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:34.625545 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:34.625550 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:34.625807 (+ 257us) client_negotiation.cc:769] Sending connection context
0812 07:13:34.626027 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:13:34.626359 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":338,"threads_started":1}
I20260812 07:13:34.627782 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:34.606796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:41763 (local address 127.21.181.126:38495)
0812 07:13:34.607324 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:34.607335 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:34.608161 (+ 826us) server_negotiation.cc:407] Connection header received
0812 07:13:34.610309 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:34.610328 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:34.610749 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:34.611129 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:34.612504 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:34.614227 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:34.615747 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:34.616709 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:34.616872 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:34.620234 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:34.620241 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:34.620253 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:34.620295 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:34.623554 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:34.624265 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:34.624274 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:34.624282 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:34.624494 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:34.624858 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:34.624862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:34.624865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:34.625426 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:34.625642 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:34.626268 (+ 626us) server_negotiation.cc:299] Negotiation successful
0812 07:13:34.626652 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":225,"threads_started":1}
I20260812 07:13:34.630475 19453 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38495
I20260812 07:13:34.630751 19453 heartbeater.cc:464] Registering TS with master...
I20260812 07:13:34.631404 19453 heartbeater.cc:511] Master 127.21.181.126:38495 requested a full tablet report, sending...
I20260812 07:13:34.633735 19119 ts_manager.cc:194] Registered new tserver with Master: e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987)
I20260812 07:13:34.635881 19119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:41763
W20260812 07:13:34.938130 19460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:34.938376 19460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:34.938521 19460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:34.945484 19460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:34.945588 19460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:34.945737 19460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:13:34.954872 19460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:35543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=37309
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:34.956017 19460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:34.957267 19460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:34.969415 19470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:34.969460 19476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:34.969481 19469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:34.970187 19460 server_base.cc:1034] running on GCE node
I20260812 07:13:34.970739 19460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:34.971606 19460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:34.972796 19460 hybrid_clock.cc:648] HybridClock initialized: now 1786518814972748 us; error 41 us; skew 500 ppm
I20260812 07:13:34.973079 19460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:34 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518814, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:34.975929 19460 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:13:34.977278 19460 webserver.cc:460] Webserver started at http://127.21.181.66:37309/ using document root <none> and password file <none>
I20260812 07:13:34.977876 19460 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:34.977990 19460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:34.982905 19460 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260812 07:13:34.985430 19491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:34.986617 19460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:13:34.986750 19460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "7dd47abc8b7249f2959cfa0f04cf164a"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "018187236c8f12cc36640fc2ee5d15a7"
server_key_iv: "07b9663cc3b955c763a8030357b9bd9a"
server_key_version: "encryptionkey@0"
I20260812 07:13:34.987627 19460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:34.998750 19460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:35.002244 19460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:35.002450 19460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:35.003283 19460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:35.004536 19460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:35.004628 19460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:35.004703 19460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:35.004756 19460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:35.034881 19460 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:35543
I20260812 07:13:35.034946 19678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:35543 every 8 connection(s)
I20260812 07:13:35.036290 19460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:13:35.039165 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 19460
I20260812 07:13:35.039754 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:38549
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:38495
--webserver_port=39771
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 12 07:13:35 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518814, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:35.054771 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.038578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:42045 (local address 127.21.181.126:38495)
0812 07:13:35.038796 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.038801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.040021 (+ 1220us) server_negotiation.cc:407] Connection header received
0812 07:13:35.041922 (+ 1901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.041927 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.042020 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.042111 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:35.043592 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.044350 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.045858 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.046283 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.046395 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.049604 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:35.049609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:35.049613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:35.049646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:35.051947 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.052639 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.052643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.052645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.052765 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.053174 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.053178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.053180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.053441 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:35.053580 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.054220 (+ 640us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.054450 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260812 07:13:35.055605 19684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.039060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:42045)
0812 07:13:35.039852 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.039906 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.041663 (+ 1757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.042350 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.042358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.042883 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:35.043412 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.043429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.044516 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.044519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.045710 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.045723 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.046515 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.046523 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.046730 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.047329 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:35.047350 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:35.049381 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:35.052103 (+ 2722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:35.052112 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:35.052126 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:35.052527 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:35.052870 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:35.052875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:35.052879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:35.053058 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:35.053577 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:35.053584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:35.053873 (+ 289us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.054093 (+ 220us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.054562 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":316,"threads_started":1}
I20260812 07:13:35.056957 19681 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38495
I20260812 07:13:35.057301 19681 heartbeater.cc:464] Registering TS with master...
I20260812 07:13:35.058069 19681 heartbeater.cc:511] Master 127.21.181.126:38495 requested a full tablet report, sending...
I20260812 07:13:35.059585 19119 ts_manager.cc:194] Registered new tserver with Master: 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66:35543)
I20260812 07:13:35.060621 19119 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:42045
W20260812 07:13:35.365547 19687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:35.365809 19687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:35.365921 19687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:35.372954 19687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:35.373062 19687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:35.373250 19687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:13:35.382717 19687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:38549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=39771
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:35.383872 19687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:35.385125 19687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:35.397557 19698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:35.397591 19697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:35.397691 19700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:35.398608 19687 server_base.cc:1034] running on GCE node
I20260812 07:13:35.399186 19687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:35.400034 19687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:35.401283 19687 hybrid_clock.cc:648] HybridClock initialized: now 1786518815401236 us; error 41 us; skew 500 ppm
I20260812 07:13:35.401588 19687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:35 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518815, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:35.404662 19687 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:13:35.406061 19687 webserver.cc:460] Webserver started at http://127.21.181.67:39771/ using document root <none> and password file <none>
I20260812 07:13:35.406690 19687 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:35.406843 19687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:35.412067 19687 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:13:35.414664 19709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:35.415894 19687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:13:35.416066 19687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "107e3873958dcd95fefbcbcc8497d2c0"
server_key_iv: "f92bb717958a6bdc8e956c99c25a41f9"
server_key_version: "encryptionkey@0"
I20260812 07:13:35.417012 19687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:35.432809 19687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:35.436728 19687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:35.436961 19687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:35.437863 19687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:35.439050 19687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:35.439154 19687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:35.439260 19687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:35.439328 19687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:35.469029 19687 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38549
I20260812 07:13:35.469089 19902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38549 every 8 connection(s)
I20260812 07:13:35.470403 19687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:13:35.478995 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 19687
Aug 12 07:13:35 dist-test-slave-lgr5 krb5kdc[18050](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518815, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:13:35.491490 19457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.472782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:52261 (local address 127.21.181.126:38495)
0812 07:13:35.472944 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.472949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.474065 (+ 1116us) server_negotiation.cc:407] Connection header received
0812 07:13:35.475920 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.475927 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.476008 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.476116 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:35.477611 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.478357 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.480551 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.481065 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.481163 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.485154 (+ 3991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:35.485160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:35.485163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:35.485194 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:35.488387 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.489111 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.489117 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.489121 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.489265 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.489714 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.489717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.489719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.489984 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:35.490096 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.490933 (+ 837us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.491172 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":52}
I20260812 07:13:35.492679 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.473225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:52261)
0812 07:13:35.473906 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.473946 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.475630 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.476371 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.476378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.476911 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:13:35.477436 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.477453 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.478537 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.478542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.480341 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.480359 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.481450 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.481455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.481665 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.482288 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:13:35.482308 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:13:35.484956 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:13:35.488517 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:35.488523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:35.488534 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:35.488965 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:35.489378 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:13:35.489383 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:13:35.489387 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:13:35.489606 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:13:35.490107 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:13:35.490115 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:13:35.490513 (+ 398us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.490801 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.491292 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":255,"threads_started":1}
I20260812 07:13:35.494017 19904 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38495
I20260812 07:13:35.494305 19904 heartbeater.cc:464] Registering TS with master...
I20260812 07:13:35.494967 19904 heartbeater.cc:511] Master 127.21.181.126:38495 requested a full tablet report, sending...
I20260812 07:13:35.495594 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.480063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53856 (local address 127.21.181.126:38495)
0812 07:13:35.480867 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.480873 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.480890 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:13:35.480932 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.480935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.481007 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.481092 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:35.482849 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.483869 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.485347 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.485814 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.485931 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.489200 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:35.489205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:35.489208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:35.489238 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:35.492721 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.493567 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.493572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.493575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.493682 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.494051 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.494055 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.494057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.494332 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:35.494532 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.494998 (+ 466us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.495205 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":688,"spinlock_wait_cycles":5504,"thread_start_us":533,"threads_started":1}
I20260812 07:13:35.496510 19118 ts_manager.cc:194] Registered new tserver with Master: 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67:38549)
I20260812 07:13:35.497349 19118 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:52261
I20260812 07:13:35.499423 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:13:35.505124 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.500234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.21.181.126:38495)
0812 07:13:35.500381 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.500384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.500856 (+ 472us) server_negotiation.cc:407] Connection header received
0812 07:13:35.500997 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.501000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.501123 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.501228 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:35.501576 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.502216 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.503604 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.504007 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.504127 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.504551 (+ 424us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:35.504708 (+ 157us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0812 07:13:35.504953 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53858: token expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260812 07:13:35.505221 19915 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53858: token expired
W20260812 07:13:35.505735 18955 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.21.181.126:38495)
I20260812 07:13:35.517299 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.506119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53874 (local address 127.21.181.126:38495)
0812 07:13:35.506281 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.506286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.506371 (+ 85us) server_negotiation.cc:407] Connection header received
0812 07:13:35.506544 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.506548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.506597 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.506668 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:35.508072 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.508758 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.510059 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.510435 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.510551 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.512876 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:35.512880 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:35.512883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:35.512910 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:35.514945 (+ 2035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.515523 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.515528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.515531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.515648 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.515988 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.515993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.515996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.516317 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:35.516538 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.516823 (+ 285us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.517016 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:13:35.520172 19118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:13:35.527099 19118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:13:35.559625 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.539464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:35543 (local address 127.0.0.1:48978)
0812 07:13:35.552069 (+ 12605us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.552112 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.552216 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.552877 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.552882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.552907 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:35.553426 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.553440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.554903 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.554908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.557035 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.557048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.558628 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.558635 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.558819 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.558870 (+ 51us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.558977 (+ 107us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.559160 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12471,"mutex_wait_us":72,"thread_start_us":11646,"threads_started":1}
I20260812 07:13:35.560164 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.537963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49920)
0812 07:13:35.550346 (+ 12383us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.550399 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.550568 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.553177 (+ 2609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.553181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.553201 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:35.553684 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.553696 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.555484 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.555489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.557659 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.557668 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.559323 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.559330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.559506 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.559542 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.559641 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.559758 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12119,"mutex_wait_us":3,"thread_start_us":11996,"threads_started":1}
I20260812 07:13:35.560351 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.540098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48978 (local address 127.21.181.66:35543)
0812 07:13:35.546806 (+ 6708us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.546815 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.552326 (+ 5511us) server_negotiation.cc:407] Connection header received
0812 07:13:35.552427 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.552435 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.552660 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.552837 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:35.553660 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.554748 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.557202 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.558402 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.558545 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.559765 (+ 1220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.559838 (+ 73us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.560023 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6545,"thread_start_us":6409,"threads_started":1}
I20260812 07:13:35.560454 19935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.537962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38549 (local address 127.0.0.1:35668)
0812 07:13:35.551440 (+ 13478us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.551463 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.551587 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.552616 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.552623 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.552655 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:35.553139 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.553153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.555205 (+ 2052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.555211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.557650 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.557666 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.559650 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.559657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.559840 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.559876 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.559958 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.560075 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13341,"mutex_wait_us":301,"thread_start_us":13659,"threads_started":1}
I20260812 07:13:35.560913 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.538379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.21.181.65:43987)
0812 07:13:35.552512 (+ 14133us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.552520 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.552546 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:13:35.552634 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.552640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.552852 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.553057 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:35.553849 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.555329 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.557828 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.558845 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.558976 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.559155 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.560420 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.560603 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13993,"thread_start_us":13889,"threads_started":1}
I20260812 07:13:35.561020 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.538048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35668 (local address 127.21.181.67:38549)
0812 07:13:35.551089 (+ 13041us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.551097 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.552025 (+ 928us) server_negotiation.cc:407] Connection header received
0812 07:13:35.552152 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.552160 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.552395 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.552585 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:35.553438 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.555021 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.557804 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.559127 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.559245 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.559459 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.560611 (+ 1152us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.560765 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12875,"thread_start_us":12733,"threads_started":1}
I20260812 07:13:35.563657 19564 tablet_service.cc:1467] Processing CreateTablet for tablet d092e7e6ade3451ebb7f46ce1b2a2683 (DEFAULT_TABLE table=test-table [id=6e28d819b5394a53a22983d844df5018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:13:35.564319 19795 tablet_service.cc:1467] Processing CreateTablet for tablet d092e7e6ade3451ebb7f46ce1b2a2683 (DEFAULT_TABLE table=test-table [id=6e28d819b5394a53a22983d844df5018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:13:35.564314 19336 tablet_service.cc:1467] Processing CreateTablet for tablet d092e7e6ade3451ebb7f46ce1b2a2683 (DEFAULT_TABLE table=test-table [id=6e28d819b5394a53a22983d844df5018]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:13:35.565141 19564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d092e7e6ade3451ebb7f46ce1b2a2683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:35.565634 19336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d092e7e6ade3451ebb7f46ce1b2a2683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:35.565963 19795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d092e7e6ade3451ebb7f46ce1b2a2683. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:13:35.576304 19937 tablet_bootstrap.cc:492] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Bootstrap starting.
I20260812 07:13:35.579602 19937 tablet_bootstrap.cc:654] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:35.580663 19938 tablet_bootstrap.cc:492] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Bootstrap starting.
I20260812 07:13:35.580844 19937 log.cc:824] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:35.581552 19939 tablet_bootstrap.cc:492] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Bootstrap starting.
I20260812 07:13:35.585148 19939 tablet_bootstrap.cc:654] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:35.585148 19938 tablet_bootstrap.cc:654] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Neither blocks nor log segments found. Creating new log.
I20260812 07:13:35.585523 19937 tablet_bootstrap.cc:492] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: No bootstrap required, opened a new log
I20260812 07:13:35.585801 19937 ts_tablet_manager.cc:1397] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260812 07:13:35.586370 19938 log.cc:824] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:35.586370 19939 log.cc:824] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:35.589030 19939 tablet_bootstrap.cc:492] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: No bootstrap required, opened a new log
I20260812 07:13:35.589270 19939 ts_tablet_manager.cc:1397] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260812 07:13:35.589427 19938 tablet_bootstrap.cc:492] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: No bootstrap required, opened a new log
I20260812 07:13:35.589700 19938 ts_tablet_manager.cc:1397] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260812 07:13:35.596221 19937 raft_consensus.cc:348] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.596642 19937 raft_consensus.cc:374] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:35.596733 19937 raft_consensus.cc:729] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dd47abc8b7249f2959cfa0f04cf164a, State: Initialized, Role: FOLLOWER
I20260812 07:13:35.597378 19937 consensus_queue.cc:260] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.597508 19939 raft_consensus.cc:348] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.597873 19939 raft_consensus.cc:374] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:35.597980 19939 raft_consensus.cc:729] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94a0fd14d7814ddb84555a5c5b9a1a6b, State: Initialized, Role: FOLLOWER
I20260812 07:13:35.598584 19681 heartbeater.cc:503] Master 127.21.181.126:38495 was elected leader, sending a full tablet report...
I20260812 07:13:35.598680 19939 consensus_queue.cc:260] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.599159 19937 ts_tablet_manager.cc:1428] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260812 07:13:35.599618 19938 raft_consensus.cc:348] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.600037 19938 raft_consensus.cc:374] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:13:35.600159 19938 raft_consensus.cc:729] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3de0038337543bea06d86bb1a0ef5dd, State: Initialized, Role: FOLLOWER
I20260812 07:13:35.600281 19904 heartbeater.cc:503] Master 127.21.181.126:38495 was elected leader, sending a full tablet report...
I20260812 07:13:35.600817 19938 consensus_queue.cc:260] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.601557 19953 raft_consensus.cc:482] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:13:35.601861 19953 raft_consensus.cc:504] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.604779 19953 leader_election.cc:288] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987), 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66:35543)
I20260812 07:13:35.612731 19939 ts_tablet_manager.cc:1428] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Time spent starting tablet: real 0.023s user 0.010s sys 0.000s
I20260812 07:13:35.615605 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.603956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.21.181.67:54951)
0812 07:13:35.604283 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.604303 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.604409 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.608961 (+ 4552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.608966 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.608995 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:35.609219 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.609230 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.610732 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.610739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.612890 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.612908 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.614761 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.614767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.615005 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.615041 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.615149 (+ 108us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.615290 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260812 07:13:35.616312 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.608315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:54951 (local address 127.21.181.65:43987)
0812 07:13:35.608515 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.608521 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.608539 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:13:35.608607 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.608613 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.608688 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.608780 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:35.609414 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.610544 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.613102 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.614430 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.614587 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.614666 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.615875 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.616020 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:13:35.617179 19369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d092e7e6ade3451ebb7f46ce1b2a2683" candidate_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3de0038337543bea06d86bb1a0ef5dd" is_pre_election: true
I20260812 07:13:35.617776 19369 raft_consensus.cc:2454] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94a0fd14d7814ddb84555a5c5b9a1a6b in term 0.
I20260812 07:13:35.618552 19712 leader_election.cc:302] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94a0fd14d7814ddb84555a5c5b9a1a6b, e3de0038337543bea06d86bb1a0ef5dd; no voters:
I20260812 07:13:35.619011 19953 raft_consensus.cc:2781] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:13:35.619184 19953 raft_consensus.cc:482] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:13:35.619315 19953 raft_consensus.cc:3037] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:35.619890 19453 heartbeater.cc:503] Master 127.21.181.126:38495 was elected leader, sending a full tablet report...
I20260812 07:13:35.620595 19938 ts_tablet_manager.cc:1428] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Time spent starting tablet: real 0.031s user 0.016s sys 0.014s
I20260812 07:13:35.622093 19953 raft_consensus.cc:504] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.622844 19953 leader_election.cc:288] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [CANDIDATE]: Term 1 election: Requested vote from peers e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987), 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66:35543)
I20260812 07:13:35.624310 19369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d092e7e6ade3451ebb7f46ce1b2a2683" candidate_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3de0038337543bea06d86bb1a0ef5dd"
I20260812 07:13:35.624529 19369 raft_consensus.cc:3037] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:35.627064 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.605583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:42457 (local address 127.21.181.66:35543)
0812 07:13:35.605766 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.605773 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.618938 (+ 13165us) server_negotiation.cc:407] Connection header received
0812 07:13:35.620226 (+ 1288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.620231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.620301 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.620357 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:13:35.621286 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.622280 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.624723 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.625918 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.626031 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.626098 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.626684 (+ 586us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.626822 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:13:35.627118 19956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.605174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:35543 (local address 127.21.181.67:42457)
0812 07:13:35.618842 (+ 13668us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.618861 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:13:35.620035 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:13:35.620550 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:13:35.620554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:13:35.620578 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:13:35.621103 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.621119 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.622439 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.622445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.624555 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:13:35.624570 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.626275 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.626281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:13:35.626468 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.626512 (+ 44us) client_negotiation.cc:769] Sending connection context
0812 07:13:35.626642 (+ 130us) client_negotiation.cc:240] Negotiation successful
0812 07:13:35.626770 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13540,"thread_start_us":13802,"threads_started":1}
I20260812 07:13:35.627740 19369 raft_consensus.cc:2454] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94a0fd14d7814ddb84555a5c5b9a1a6b in term 1.
I20260812 07:13:35.628278 19600 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d092e7e6ade3451ebb7f46ce1b2a2683" candidate_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd47abc8b7249f2959cfa0f04cf164a"
I20260812 07:13:35.628415 19712 leader_election.cc:302] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94a0fd14d7814ddb84555a5c5b9a1a6b, e3de0038337543bea06d86bb1a0ef5dd; no voters:
I20260812 07:13:35.628665 19600 raft_consensus.cc:3037] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:13:35.628813 19953 raft_consensus.cc:2781] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:13:35.629172 19953 raft_consensus.cc:686] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 LEADER]: Becoming Leader. State: Replica: 94a0fd14d7814ddb84555a5c5b9a1a6b, State: Running, Role: LEADER
I20260812 07:13:35.629796 19953 consensus_queue.cc:237] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } }
I20260812 07:13:35.631832 19600 raft_consensus.cc:2454] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94a0fd14d7814ddb84555a5c5b9a1a6b in term 1.
I20260812 07:13:35.628021 19601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d092e7e6ade3451ebb7f46ce1b2a2683" candidate_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" is_pre_election: true
I20260812 07:13:35.632143 19601 raft_consensus.cc:2362] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 94a0fd14d7814ddb84555a5c5b9a1a6b in term 1. Re-sending same reply.
I20260812 07:13:35.633940 19118 catalog_manager.cc:5495] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67). New cstate: current_term: 1 leader_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 } health_report { overall_health: UNKNOWN } } }
I20260812 07:13:35.656351 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.651092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.21.181.67:38549)
0812 07:13:35.651263 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.651269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.651386 (+ 117us) server_negotiation.cc:407] Connection header received
0812 07:13:35.651620 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.651624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.651700 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.651819 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:13:35.652242 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.652929 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.654329 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.654714 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.654820 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.655326 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:13:35.655772 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:13:35.655898 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.656019 (+ 121us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.656122 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:13:35.663694 19369 raft_consensus.cc:1261] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Refusing update from remote peer 94a0fd14d7814ddb84555a5c5b9a1a6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:35.663707 19600 raft_consensus.cc:1261] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Refusing update from remote peer 94a0fd14d7814ddb84555a5c5b9a1a6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:13:35.664641 19954 consensus_queue.cc:1035] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7dd47abc8b7249f2959cfa0f04cf164a" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 35543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:35.665002 19953 consensus_queue.cc:1035] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3de0038337543bea06d86bb1a0ef5dd" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 43987 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:13:35.689033 19118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:13:35.689263 19118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:13:35.692559 19118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Sending DeleteTablet for 3 replicas of tablet d092e7e6ade3451ebb7f46ce1b2a2683
I20260812 07:13:35.693475 19795 tablet_service.cc:1514] Processing DeleteTablet for tablet d092e7e6ade3451ebb7f46ce1b2a2683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:13:35 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:35668
I20260812 07:13:35.693704 19336 tablet_service.cc:1514] Processing DeleteTablet for tablet d092e7e6ade3451ebb7f46ce1b2a2683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:13:35 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:49920
I20260812 07:13:35.694280 19564 tablet_service.cc:1514] Processing DeleteTablet for tablet d092e7e6ade3451ebb7f46ce1b2a2683 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:13:35 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:48978
I20260812 07:13:35.695086 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 19186
I20260812 07:13:35.709337 19984 tablet_replica.cc:330] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: stopping tablet replica
I20260812 07:13:35.709682 19984 raft_consensus.cc:2229] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:13:35.711055 19984 raft_consensus.cc:2258] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:13:35.713871 19984 ts_tablet_manager.cc:1905] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:35.717713 19985 tablet_replica.cc:330] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: stopping tablet replica
I20260812 07:13:35.718026 19932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.696690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.21.181.65:43987)
0812 07:13:35.696851 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.696857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.696875 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:13:35.696929 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.696934 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.697029 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.697122 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:35.699071 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.700276 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.702206 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.702866 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.703008 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.710020 (+ 7012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:35.710034 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:35.710051 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:35.710093 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:35.713936 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.714828 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.714837 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.714840 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.715007 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.715487 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.715492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.715495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.715981 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:35.716304 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.717355 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.717590 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:13:35.718127 19985 raft_consensus.cc:2229] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:13:35.719018 19442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:35.719086 19985 raft_consensus.cc:2258] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:13:35.719790 19983 tablet_replica.cc:330] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: stopping tablet replica
I20260812 07:13:35.721968 19985 ts_tablet_manager.cc:1905] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:35.728140 19983 raft_consensus.cc:2229] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:13:35.728700 19985 ts_tablet_manager.cc:1918] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:35.728854 19985 log.cc:1196] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/d092e7e6ade3451ebb7f46ce1b2a2683
I20260812 07:13:35.728982 19983 raft_consensus.cc:2258] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:13:35.729406 19985 ts_tablet_manager.cc:1939] T d092e7e6ade3451ebb7f46ce1b2a2683 P 7dd47abc8b7249f2959cfa0f04cf164a: Deleting consensus metadata
I20260812 07:13:35.731140 19983 ts_tablet_manager.cc:1905] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:35.731164 19091 catalog_manager.cc:4841] TS 7dd47abc8b7249f2959cfa0f04cf164a (127.21.181.66:35543): tablet d092e7e6ade3451ebb7f46ce1b2a2683 (table test-table [id=6e28d819b5394a53a22983d844df5018]) successfully deleted
I20260812 07:13:35.738739 19983 ts_tablet_manager.cc:1918] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:35.738920 19983 log.cc:1196] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/d092e7e6ade3451ebb7f46ce1b2a2683
I20260812 07:13:35.739751 19983 ts_tablet_manager.cc:1939] T d092e7e6ade3451ebb7f46ce1b2a2683 P 94a0fd14d7814ddb84555a5c5b9a1a6b: Deleting consensus metadata
I20260812 07:13:35.741935 19094 catalog_manager.cc:4841] TS 94a0fd14d7814ddb84555a5c5b9a1a6b (127.21.181.67:38549): tablet d092e7e6ade3451ebb7f46ce1b2a2683 (table test-table [id=6e28d819b5394a53a22983d844df5018]) successfully deleted
I20260812 07:13:35.868745 19984 ts_tablet_manager.cc:1918] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:35.869014 19984 log.cc:1196] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/d092e7e6ade3451ebb7f46ce1b2a2683
I20260812 07:13:35.869416 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 19186
I20260812 07:13:35.869535 19984 ts_tablet_manager.cc:1939] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Deleting consensus metadata
W20260812 07:13:35.893227 19092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.181.65:43987 (error 108)
I20260812 07:13:35.893399 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 19460
I20260812 07:13:35.895133 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.894660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49944)
0812 07:13:35.894836 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.894947 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:13:35.895464 19092 catalog_manager.cc:4584] TS e3de0038337543bea06d86bb1a0ef5dd (127.21.181.65:43987): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d092e7e6ade3451ebb7f46ce1b2a2683: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
I20260812 07:13:35.908840 19936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.894189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48980 (local address 127.21.181.66:35543)
0812 07:13:35.894783 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:35.894789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:35.894808 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:13:35.894875 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:35.894880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:35.894960 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:35.895101 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:35.896903 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.897657 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.899507 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:35.899889 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:35.899976 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:35.903202 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:35.903217 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:35.903231 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:35.903262 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:35.906272 (+ 3010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.907057 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.907064 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.907066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.907177 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:35.907524 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:35.907526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:35.907528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:35.907865 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:35.908203 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:35.908404 (+ 201us) server_negotiation.cc:299] Negotiation successful
0812 07:13:35.908560 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467}
I20260812 07:13:35.909531 19671 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:35.949203 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.948616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49946)
0812 07:13:35.948861 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.949008 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260812 07:13:35.950439 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:35.950008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49952)
0812 07:13:35.950173 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:35.950289 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:13:35.999060 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 19460
I20260812 07:13:36.011561 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:36.011120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49962)
0812 07:13:36.011317 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:36.011405 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:13:36.012603 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:36.012274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49978)
0812 07:13:36.012417 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:36.012486 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 07:13:36.034050 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 19687
I20260812 07:13:36.050277 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:36.034856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35686 (local address 127.21.181.67:38549)
0812 07:13:36.035024 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:13:36.035029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:13:36.035057 (+ 28us) server_negotiation.cc:407] Connection header received
0812 07:13:36.035211 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:13:36.035214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:13:36.035285 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:13:36.035364 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:13:36.037330 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:36.038161 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:36.040298 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:13:36.040715 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:13:36.040803 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:13:36.044329 (+ 3526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:13:36.044341 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:13:36.044361 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:13:36.044402 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:13:36.047257 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:36.048141 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:36.048148 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:36.048150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:36.048316 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:13:36.048734 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:13:36.048740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:13:36.048743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:13:36.049156 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:13:36.049389 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:13:36.049760 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 07:13:36.049918 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:13:36.051038 19895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:36.093046 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:36.092627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49990)
0812 07:13:36.092813 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:36.092897 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:13:36.094013 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:36.093643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:49998)
0812 07:13:36.093832 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:36.093903 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260812 07:13:36.171957 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 19687
I20260812 07:13:36.202189 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 19056
I20260812 07:13:36.203326 19172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:13:36.313616 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:36.313027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:43987 (local address 127.0.0.1:50000)
0812 07:13:36.313302 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:36.313407 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:43987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260812 07:13:36.313694 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 19056
I20260812 07:13:56.342396 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38495
--webserver_interface=127.21.181.126
--webserver_port=45773
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:38495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:13:56.683588 19993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:56.683854 19993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:56.683981 19993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:56.690908 19993 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260812 07:13:56.690991 19993 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260812 07:13:56.691046 19993 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260812 07:13:56.691124 19993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:13:56.691187 19993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:56.691280 19993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:56.691339 19993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:13:56.691411 19993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:13:56.700801 19993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:38495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=45773
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:56.702201 19993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:56.703534 19993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:56.714372 20002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:56.714905 20004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:56.714931 20006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:56.715543 19993 server_base.cc:1034] running on GCE node
I20260812 07:13:56.716295 19993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:56.717535 19993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:56.718750 19993 hybrid_clock.cc:648] HybridClock initialized: now 1786518836718686 us; error 40 us; skew 500 ppm
I20260812 07:13:56.719048 19993 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:56 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518836, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:56.724889 19993 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:13:56.726270 19993 webserver.cc:460] Webserver started at http://127.21.181.126:45773/ using document root <none> and password file <none>
I20260812 07:13:56.726950 19993 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:56.727079 19993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:56.732465 19993 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:13:56.735137 20021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:56.736371 19993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:13:56.736546 19993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "19a26a82f6ba49d48da1737da46a74ba"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "d6d1a87c694996831578ae5b05eaf91b"
server_key_iv: "486792be26dc17030f0c16fa14607e1f"
server_key_version: "encryptionkey@0"
I20260812 07:13:56.737419 19993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:56.760456 19993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:56.764007 19993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:56.764271 19993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:56.781385 19993 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:38495
I20260812 07:13:56.781425 20100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:38495 every 8 connection(s)
I20260812 07:13:56.782752 19993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:13:56.784241 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 19993
I20260812 07:13:56.785270 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:43987
--local_ip_for_outbound_sockets=127.21.181.65
--tserver_master_addrs=127.21.181.126:38495
--webserver_port=35387
--webserver_interface=127.21.181.65
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:56.791278 20101 sys_catalog.cc:263] Verifying existing consensus state
I20260812 07:13:56.795463 20101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap starting.
I20260812 07:13:56.815686 20101 log.cc:824] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Log is configured to *not* fsync() on all Append() calls
I20260812 07:13:56.831563 20101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 07:13:56.832054 20101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Bootstrap complete.
I20260812 07:13:56.841398 20101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:56.842278 20101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19a26a82f6ba49d48da1737da46a74ba, State: Initialized, Role: FOLLOWER
I20260812 07:13:56.842839 20101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:56.843056 20101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:13:56.843164 20101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:13:56.843322 20101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 2 FOLLOWER]: Advancing to term 3
I20260812 07:13:56.847456 20101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:56.847906 20101 leader_election.cc:302] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19a26a82f6ba49d48da1737da46a74ba; no voters:
I20260812 07:13:56.848594 20101 leader_election.cc:288] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [CANDIDATE]: Term 3 election: Requested vote from peers
I20260812 07:13:56.848724 20107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 3 FOLLOWER]: Leader election won for term 3
I20260812 07:13:56.850572 20101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:13:56.853874 20107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [term 3 LEADER]: Becoming Leader. State: Replica: 19a26a82f6ba49d48da1737da46a74ba, State: Running, Role: LEADER
I20260812 07:13:56.854521 20107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } }
I20260812 07:13:56.857420 20109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "19a26a82f6ba49d48da1737da46a74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } } }
I20260812 07:13:56.857786 20109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:56.858296 20111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19a26a82f6ba49d48da1737da46a74ba. Latest consensus state: current_term: 3 leader_uuid: "19a26a82f6ba49d48da1737da46a74ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19a26a82f6ba49d48da1737da46a74ba" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38495 } } }
I20260812 07:13:56.858598 20111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba [sys.catalog]: This master's current role is: LEADER
I20260812 07:13:56.859836 20137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:13:56.869225 20137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:13:56.870886 20137 catalog_manager.cc:1259] Loaded cluster ID: 09b9e69760534fd6853fb8ece1a66b96
I20260812 07:13:56.871042 20137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:13:56.875349 20137 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:13:56.877111 20137 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Loaded TSK: 0
I20260812 07:13:56.877336 20137 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 19a26a82f6ba49d48da1737da46a74ba: Loaded TSK: 1
F20260812 07:13:56.878614 20137 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb2e59eb9ad google::LogMessage::Fail() at ??:0
@ 0x7fb2e59ee88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb2e59eb4de google::LogMessage::Flush() at ??:0
@ 0x7fb2e59eca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb2e974f7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fb2e973e73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fb2fb41a1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
I20260812 07:13:56.901274 20157 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7fb2fb419783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fb2fb4a52a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fb2fb4a50c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fb2fb4ef5e9 std::function<>::operator()() at ??:0
@ 0x7fb2fb41dcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb2fb41bc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb2fb4a3610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb2fb4a3439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb2e8637345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb2e863a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb2e863a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb2e8615eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb2eb8ef6db start_thread at ??:0
@ 0x7fb2e488561f clone at ??:0
W20260812 07:13:57.152139 20105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:57.152423 20105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:57.152562 20105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:57.159668 20105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:57.159782 20105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:57.159932 20105 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:13:57.169426 20105 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:43987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=35387
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:57.170603 20105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:57.171844 20105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:57.184470 20200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:57.184576 20201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:57.184603 20203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:57.185470 20105 server_base.cc:1034] running on GCE node
I20260812 07:13:57.186033 20105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:57.186887 20105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:57.188095 20105 hybrid_clock.cc:648] HybridClock initialized: now 1786518837188034 us; error 46 us; skew 500 ppm
I20260812 07:13:57.188392 20105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:57 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518837, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:57.191221 20105 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:13:57.192601 20105 webserver.cc:460] Webserver started at http://127.21.181.65:35387/ using document root <none> and password file <none>
I20260812 07:13:57.193233 20105 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:57.193352 20105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:57.198323 20105 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260812 07:13:57.200846 20213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:57.202008 20105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:13:57.202144 20105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "e3de0038337543bea06d86bb1a0ef5dd"
format_stamp: "Formatted at 2026-08-12 07:13:11 on dist-test-slave-lgr5"
server_key: "43f49456d6f7f2e466d20f96191441b9"
server_key_iv: "c2ff5464d5239754886de75f2b5c4e69"
server_key_version: "encryptionkey@0"
I20260812 07:13:57.203033 20105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:57.219457 20105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:57.222882 20105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:57.223114 20105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:57.223976 20105 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:57.225566 20224 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260812 07:13:57.230466 20224 ts_tablet_manager.cc:1852] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260812 07:13:57.231408 20224 ts_tablet_manager.cc:1905] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:13:57.238198 20224 ts_tablet_manager.cc:1918] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:13:57.238446 20224 ts_tablet_manager.cc:1939] T d092e7e6ade3451ebb7f46ce1b2a2683 P e3de0038337543bea06d86bb1a0ef5dd: Deleting consensus metadata
I20260812 07:13:57.238895 20105 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260812 07:13:57.239010 20105 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.014s user 0.001s sys 0.000s
I20260812 07:13:57.239118 20105 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:57.239192 20105 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:57.269996 20105 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:43987
I20260812 07:13:57.270026 20400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:43987 every 8 connection(s)
I20260812 07:13:57.271209 20105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:13:57.275705 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 20105
I20260812 07:13:57.275772 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:57.273964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:45173)
0812 07:13:57.274684 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:57.274913 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":281,"threads_started":1}
W20260812 07:13:57.276376 20401 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:38495 (0 consecutive failures): Network error: Failed to ping master at 127.21.181.126:38495: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
I20260812 07:13:57.276536 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:35543
--local_ip_for_outbound_sockets=127.21.181.66
--tserver_master_addrs=127.21.181.126:38495
--webserver_port=37309
--webserver_interface=127.21.181.66
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:57.277169 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:57.276869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:58819)
0812 07:13:57.277008 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:57.277072 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:13:57.277935 20405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:57.277642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:53421)
0812 07:13:57.277755 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:57.277824 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:13:57.278141 20401 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:13:57.597024 20406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:57.597268 20406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:57.597401 20406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:57.604323 20406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:57.604419 20406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:57.604578 20406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:13:57.613761 20406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:35543
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=37309
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:57.614861 20406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:57.616067 20406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:57.630105 20412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:57.630071 20414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:57.630031 20417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:57.630771 20406 server_base.cc:1034] running on GCE node
I20260812 07:13:57.631351 20406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:57.632280 20406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:57.633487 20406 hybrid_clock.cc:648] HybridClock initialized: now 1786518837633408 us; error 60 us; skew 500 ppm
I20260812 07:13:57.633772 20406 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:57 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518837, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:57.636699 20406 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:13:57.638046 20406 webserver.cc:460] Webserver started at http://127.21.181.66:37309/ using document root <none> and password file <none>
I20260812 07:13:57.638692 20406 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:57.638808 20406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:57.643774 20406 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:13:57.646266 20434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:57.647437 20406 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:13:57.647599 20406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "7dd47abc8b7249f2959cfa0f04cf164a"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "018187236c8f12cc36640fc2ee5d15a7"
server_key_iv: "07b9663cc3b955c763a8030357b9bd9a"
server_key_version: "encryptionkey@0"
I20260812 07:13:57.648571 20406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:57.659574 20406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:57.663141 20406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:57.663370 20406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:57.664302 20406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:57.665525 20406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:57.665627 20406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:57.665724 20406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:57.665793 20406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:57.696560 20406 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:35543
I20260812 07:13:57.696620 20616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:35543 every 8 connection(s)
I20260812 07:13:57.697934 20406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:13:57.702478 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:57.700751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:38027)
0812 07:13:57.701445 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:57.701671 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":264,"threads_started":1}
W20260812 07:13:57.703048 20617 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:38495 (0 consecutive failures): Network error: Failed to ping master at 127.21.181.126:38495: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
I20260812 07:13:57.703820 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:57.703460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:53135)
0812 07:13:57.703609 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:57.703691 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260812 07:13:57.703912 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 20406
I20260812 07:13:57.704658 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:38549
--local_ip_for_outbound_sockets=127.21.181.67
--tserver_master_addrs=127.21.181.126:38495
--webserver_port=39771
--webserver_interface=127.21.181.67
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:34689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:13:57.704717 20620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:57.704353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:43287)
0812 07:13:57.704489 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:57.704580 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:13:57.704952 20617 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260812 07:13:58.031697 20621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:13:58.031931 20621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:13:58.032074 20621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:13:58.039129 20621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:13:58.039232 20621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:13:58.039384 20621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:13:58.048563 20621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:38549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=39771
--tserver_master_addrs=127.21.181.126:38495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:13:58.049664 20621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:13:58.050866 20621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:13:58.063184 20627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:58.063230 20628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:13:58.063264 20632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:13:58.063951 20621 server_base.cc:1034] running on GCE node
I20260812 07:13:58.064522 20621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:13:58.065357 20621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:13:58.066561 20621 hybrid_clock.cc:648] HybridClock initialized: now 1786518838066513 us; error 35 us; skew 500 ppm
I20260812 07:13:58.066848 20621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:13:58 dist-test-slave-lgr5 krb5kdc[18050](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518838, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:13:58.069792 20621 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:13:58.071434 20621 webserver.cc:460] Webserver started at http://127.21.181.67:39771/ using document root <none> and password file <none>
I20260812 07:13:58.072047 20621 fs_manager.cc:359] Metadata directory not provided
I20260812 07:13:58.072197 20621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:13:58.077158 20621 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:13:58.079813 20646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:58.081071 20621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:13:58.081220 20621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "94a0fd14d7814ddb84555a5c5b9a1a6b"
format_stamp: "Formatted at 2026-08-12 07:13:12 on dist-test-slave-lgr5"
server_key: "107e3873958dcd95fefbcbcc8497d2c0"
server_key_iv: "f92bb717958a6bdc8e956c99c25a41f9"
server_key_version: "encryptionkey@0"
I20260812 07:13:58.082134 20621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:13:58.102571 20621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:13:58.106436 20621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:13:58.106664 20621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:13:58.107514 20621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:13:58.108734 20621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:13:58.108842 20621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:58.108945 20621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:13:58.109021 20621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:13:58.138545 20621 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38549
I20260812 07:13:58.138607 20836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38549 every 8 connection(s)
I20260812 07:13:58.139846 20621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:13:58.141078 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 20621
I20260812 07:13:58.144719 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:58.142969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:41785)
0812 07:13:58.143669 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:58.143898 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":254,"threads_started":1}
W20260812 07:13:58.145280 20837 heartbeater.cc:644] Failed to heartbeat to 127.21.181.126:38495 (0 consecutive failures): Network error: Failed to ping master at 127.21.181.126:38495: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
I20260812 07:13:58.146023 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:58.145694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:54387)
0812 07:13:58.145840 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:58.145907 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:13:58.146802 20843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:58.146503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:36959)
0812 07:13:58.146624 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:58.146697 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:13:58.147019 20837 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260812 07:13:58.279327 20845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:58.278619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:54059)
0812 07:13:58.279040 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:58.279117 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":199,"threads_started":1}
I20260812 07:13:58.706142 20846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:58.705391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:55489)
0812 07:13:58.705826 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:58.705900 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":210,"threads_started":1}
I20260812 07:13:59.148222 20849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:59.147451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:52101)
0812 07:13:59.147874 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:59.147957 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":207,"threads_started":1}
I20260812 07:13:59.280786 20854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:59.280127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:35561)
0812 07:13:59.280549 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:59.280632 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":227,"threads_started":1}
I20260812 07:13:59.707609 20856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:13:59.706897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:56269)
0812 07:13:59.707335 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 07:13:59.707409 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":221,"threads_started":1}
I20260812 07:14:00.149636 20857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:00.148991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:49347)
0812 07:14:00.149402 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:00.149479 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":202,"threads_started":1}
I20260812 07:14:00.282294 20858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:00.281549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:50961)
0812 07:14:00.282006 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:00.282090 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":223,"threads_started":1}
I20260812 07:14:00.709036 20860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:00.708374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:43889)
0812 07:14:00.708798 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:00.708875 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":219,"threads_started":1}
I20260812 07:14:01.151093 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:01.150360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:60887)
0812 07:14:01.150820 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:01.150919 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":254,"threads_started":1}
I20260812 07:14:01.283784 20864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:01.283061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:52161)
0812 07:14:01.283517 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:01.283595 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":242,"threads_started":1}
I20260812 07:14:01.710466 20865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:01.709773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:59815)
0812 07:14:01.710201 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:01.710278 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260812 07:14:02.152650 20867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:02.151898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:54565)
0812 07:14:02.152369 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:02.152450 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":257,"threads_started":1}
I20260812 07:14:02.285223 20868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:02.284534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:53479)
0812 07:14:02.284964 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:02.285054 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":202,"threads_started":1}
I20260812 07:14:02.711922 20869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:02.711247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:52781)
0812 07:14:02.711685 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:02.711763 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260812 07:14:03.154132 20871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:03.153427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:43697)
0812 07:14:03.153890 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:03.153969 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":238,"threads_started":1}
I20260812 07:14:03.286701 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:03.285971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:57091)
0812 07:14:03.286436 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:03.286529 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":229,"threads_started":1}
I20260812 07:14:03.713356 20875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:03.712650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:42201)
0812 07:14:03.713092 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:03.713183 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":228,"threads_started":1}
I20260812 07:14:04.155709 20876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:04.154879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:54479)
0812 07:14:04.155385 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:04.155517 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":274,"threads_started":1}
I20260812 07:14:04.288237 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:04.287462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:36359)
0812 07:14:04.287921 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:04.288011 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":211,"threads_started":1}
I20260812 07:14:04.714999 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:04.714225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:36353)
0812 07:14:04.714688 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:04.714765 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":211,"threads_started":1}
I20260812 07:14:05.157339 20885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:05.156558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:34663)
0812 07:14:05.157051 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:05.157131 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":208,"threads_started":1}
I20260812 07:14:05.289600 20886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:05.288963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:46081)
0812 07:14:05.289344 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:05.289419 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":203,"threads_started":1}
I20260812 07:14:05.716466 20888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:05.715774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:47523)
0812 07:14:05.716230 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:05.716310 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":214,"threads_started":1}
I20260812 07:14:06.158784 20890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:06.158127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:38577)
0812 07:14:06.158556 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:06.158636 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":219,"threads_started":1}
I20260812 07:14:06.291031 20891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:06.290346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:43161)
0812 07:14:06.290795 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:06.290882 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":222,"threads_started":1}
I20260812 07:14:06.717934 20892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:06.717241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:57925)
0812 07:14:06.717688 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:06.717767 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":217,"threads_started":1}
I20260812 07:14:07.160408 20893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:07.159576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:40861)
0812 07:14:07.160133 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:07.160222 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":268,"threads_started":1}
I20260812 07:14:07.292547 20895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:07.291826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:57927)
0812 07:14:07.292288 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:07.292364 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":263,"threads_started":1}
I20260812 07:14:07.719457 20896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:07.718774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:50803)
0812 07:14:07.719220 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:07.719299 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":250,"threads_started":1}
I20260812 07:14:08.161890 20898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:08.161234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:53785)
0812 07:14:08.161657 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:08.161735 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":230,"threads_started":1}
I20260812 07:14:08.293993 20899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:08.293293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:45569)
0812 07:14:08.293743 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:08.293822 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":245,"threads_started":1}
I20260812 07:14:08.720963 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:08.720284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:44639)
0812 07:14:08.720710 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:08.720787 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":211,"threads_started":1}
I20260812 07:14:09.163362 20903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:09.162666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:38843)
0812 07:14:09.163090 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:09.163184 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":206,"threads_started":1}
I20260812 07:14:09.295413 20904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:09.294749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:50971)
0812 07:14:09.295161 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:09.295241 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":191,"threads_started":1}
I20260812 07:14:09.722426 20906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:09.721781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:40685)
0812 07:14:09.722190 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:09.722267 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":204,"threads_started":1}
I20260812 07:14:10.164829 20909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:10.164187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:34561)
0812 07:14:10.164598 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:10.164679 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":202,"threads_started":1}
I20260812 07:14:10.296993 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:10.296275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:50345)
0812 07:14:10.296724 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:10.296827 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260812 07:14:10.723977 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:10.723235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:38895)
0812 07:14:10.723688 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:10.723802 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":213,"threads_started":1}
I20260812 07:14:11.166345 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:11.165618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:41079)
0812 07:14:11.166070 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:11.166160 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":216,"threads_started":1}
I20260812 07:14:11.298427 20914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:11.297734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:42059)
0812 07:14:11.298160 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:11.298239 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":210,"threads_started":1}
I20260812 07:14:11.725507 20915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:11.724757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:40547)
0812 07:14:11.725238 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:11.725318 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":253,"threads_started":1}
I20260812 07:14:12.167843 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:12.167173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:46909)
0812 07:14:12.167583 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:12.167664 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":210,"threads_started":1}
I20260812 07:14:12.300137 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:12.299414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.65:38445)
0812 07:14:12.299847 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:12.299941 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":213,"threads_started":1}
I20260812 07:14:12.727157 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:12.726465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.66:60015)
0812 07:14:12.726874 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:12.726975 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":229,"threads_started":1}
I20260812 07:14:13.145962 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 20105
I20260812 07:14:13.163370 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:13.146947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37300 (local address 127.21.181.65:43987)
0812 07:14:13.147547 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:13.147565 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:13.147608 (+ 43us) server_negotiation.cc:407] Connection header received
0812 07:14:13.149307 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:13.149317 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:13.149740 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:13.150053 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:13.151646 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:13.152938 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:13.154190 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:13.154844 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:13.155022 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:13.157597 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:13.157605 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:13.157619 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:13.157656 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:13.160293 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:13.160951 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:13.160957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:13.160963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:13.161121 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:13.161487 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:13.161490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:13.161500 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:13.161985 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:13.162262 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:13.162712 (+ 450us) server_negotiation.cc:299] Negotiation successful
0812 07:14:13.163086 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":275,"threads_started":1}
I20260812 07:14:13.164685 20392 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:13.169618 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:13.168874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38495 (local address 127.21.181.67:53649)
0812 07:14:13.169335 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:13.169447 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.126:38495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":252,"threads_started":1}
I20260812 07:14:13.246829 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 20105
I20260812 07:14:13.272253 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 20406
I20260812 07:14:13.288815 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:13.273085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46180 (local address 127.21.181.66:35543)
0812 07:14:13.273622 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:13.273636 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:13.273671 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:14:13.275251 (+ 1580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:13.275261 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:13.275671 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:13.275949 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:13.277287 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:13.278506 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:13.279635 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:13.280373 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:13.280500 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:13.282977 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:13.282985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:13.283008 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:13.283052 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:13.286011 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:13.286631 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:13.286638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:13.286644 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:13.286812 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:13.287097 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:13.287100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:13.287102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:13.287571 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:13.287804 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:13.288145 (+ 341us) server_negotiation.cc:299] Negotiation successful
0812 07:14:13.288529 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":252,"threads_started":1}
I20260812 07:14:13.290118 20611 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:13.372741 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 20406
I20260812 07:14:13.397825 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 20621
I20260812 07:14:13.415205 20926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:13.398703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37284 (local address 127.21.181.67:38549)
0812 07:14:13.399269 (+ 566us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:13.399284 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:13.399324 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:14:13.400996 (+ 1672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:13.401007 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:13.401487 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:13.401805 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:13.403257 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:13.404625 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:13.405849 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:13.406514 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:13.406680 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:13.409331 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:13.409339 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:13.409353 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:13.409389 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:13.411917 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:13.412633 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:13.412641 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:13.412650 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:13.412823 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:13.413194 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:13.413198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:13.413200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:13.413734 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:13.414048 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:13.414470 (+ 422us) server_negotiation.cc:299] Negotiation successful
0812 07:14:13.414887 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":245,"threads_started":1}
I20260812 07:14:13.416587 20829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:13.497895 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 20621
2026-08-12T07:14:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64118 ms)
[----------] 30 tests from SecurityITest (290662 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:13 dist-test-slave-lgr5 krb5kdc[20938](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:13 dist-test-slave-lgr5 krb5kdc[20938](info): set up 2 sockets
Aug 12 07:14:13 dist-test-slave-lgr5 krb5kdc[20938](info): commencing operation
krb5kdc: starting...
W20260812 07:14:14.763417 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.002s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:14 dist-test-slave-lgr5 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:14:15.029225 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44697
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:44697
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:15.348591 20960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:15.348855 20960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:15.348939 20960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:15.355820 20960 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:15.355911 20960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:15.355969 20960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:15.356032 20960 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:15.356076 20960 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:15.365846 20960 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:44697
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:44697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:15.366993 20960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:15.368261 20960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:15.384123 20977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:15.384194 20971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:15.384218 20972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:15.384997 20960 server_base.cc:1034] running on GCE node
I20260812 07:14:15.385708 20960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:15.386951 20960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:15.388170 20960 hybrid_clock.cc:648] HybridClock initialized: now 1786518855388113 us; error 47 us; skew 500 ppm
I20260812 07:14:15.388485 20960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:15 dist-test-slave-lgr5 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518855, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:15.394760 20960 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:14:15.396162 20960 webserver.cc:460] Webserver started at http://127.21.181.126:36227/ using document root <none> and password file <none>
I20260812 07:14:15.396790 20960 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:15.396904 20960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:15.397184 20960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:15.399936 20960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "6d147a88c35e45a885ec47d0128c98f0"
format_stamp: "Formatted at 2026-08-12 07:14:15 on dist-test-slave-lgr5"
server_key: "016dd9e572e985a2a862b10772f2711d"
server_key_iv: "598c1872e3afd3eebc1cdb33b3c5b563"
server_key_version: "encryptionkey@0"
I20260812 07:14:15.400689 20960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "6d147a88c35e45a885ec47d0128c98f0"
format_stamp: "Formatted at 2026-08-12 07:14:15 on dist-test-slave-lgr5"
server_key: "016dd9e572e985a2a862b10772f2711d"
server_key_iv: "598c1872e3afd3eebc1cdb33b3c5b563"
server_key_version: "encryptionkey@0"
I20260812 07:14:15.405385 20960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:14:15.408500 20992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:15.409744 20960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260812 07:14:15.409893 20960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "6d147a88c35e45a885ec47d0128c98f0"
format_stamp: "Formatted at 2026-08-12 07:14:15 on dist-test-slave-lgr5"
server_key: "016dd9e572e985a2a862b10772f2711d"
server_key_iv: "598c1872e3afd3eebc1cdb33b3c5b563"
server_key_version: "encryptionkey@0"
I20260812 07:14:15.410059 20960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:15.439095 20960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:15.442680 20960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:15.442929 20960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:15.459822 20960 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:44697
I20260812 07:14:15.459830 21083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:44697 every 8 connection(s)
I20260812 07:14:15.461038 20960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:15.464515 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:15.470171 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 20960
I20260812 07:14:15.470345 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:15.470984 22229 external_mini_cluster.cc:1428] Setting key 2b47f3cf58c3af8882489b2d58d85b37
I20260812 07:14:15.474493 21084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: Bootstrap starting.
I20260812 07:14:15.477952 21084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:15.478938 21084 log.cc:824] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:15.481762 21084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: No bootstrap required, opened a new log
Aug 12 07:14:15 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:15.490247 21084 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d147a88c35e45a885ec47d0128c98f0" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44697 } }
I20260812 07:14:15.490638 21084 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:15.490758 21084 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d147a88c35e45a885ec47d0128c98f0, State: Initialized, Role: FOLLOWER
I20260812 07:14:15.491315 21084 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d147a88c35e45a885ec47d0128c98f0" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44697 } }
I20260812 07:14:15.491560 21084 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:15.491719 21084 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:15.491940 21084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:15.493786 21090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:15.472803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.21.181.126:44697)
0812 07:14:15.473518 (+ 715us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:15.473529 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:15.473568 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:14:15.475230 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:15.475252 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:15.475602 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:15.475908 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:15.477717 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:15.479033 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:15.480721 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:15.481347 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:15.481472 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:15.486206 (+ 4734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:15.486213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:15.486229 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:15.486264 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:15.489257 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:15.489961 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:15.489968 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:15.489974 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:15.490142 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:15.490496 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:15.490501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:15.490503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:15.491127 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:15.491415 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:15.491963 (+ 548us) server_negotiation.cc:299] Negotiation successful
0812 07:14:15.492424 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":528,"thread_start_us":332,"threads_started":1}
I20260812 07:14:15.495007 21084 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d147a88c35e45a885ec47d0128c98f0" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44697 } }
I20260812 07:14:15.495513 21084 leader_election.cc:302] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d147a88c35e45a885ec47d0128c98f0; no voters:
I20260812 07:14:15.496134 21084 leader_election.cc:288] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:15.496275 21095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:15.496975 21095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [term 1 LEADER]: Becoming Leader. State: Replica: 6d147a88c35e45a885ec47d0128c98f0, State: Running, Role: LEADER
I20260812 07:14:15.497541 21095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d147a88c35e45a885ec47d0128c98f0" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44697 } }
I20260812 07:14:15.498344 21084 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:15.500172 21099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d147a88c35e45a885ec47d0128c98f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d147a88c35e45a885ec47d0128c98f0" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44697 } } }
I20260812 07:14:15.500706 21099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:15.506071 21100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d147a88c35e45a885ec47d0128c98f0. Latest consensus state: current_term: 1 leader_uuid: "6d147a88c35e45a885ec47d0128c98f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d147a88c35e45a885ec47d0128c98f0" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 44697 } } }
I20260812 07:14:15.506884 21100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0 [sys.catalog]: This master's current role is: LEADER
W20260812 07:14:15.509104 21127 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:14:15.509223 21127 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:14:15.511739 21111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:15.513978 21111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:15.524330 21111 catalog_manager.cc:1347] Generated new cluster ID: c05fccca51a940019e94b84f78dd1fe8
I20260812 07:14:15.524443 21111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:15.537050 21111 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:15.538854 21111 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:15.554585 21111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: Generated new TSK 0
I20260812 07:14:15.555545 21111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:14:15.651249 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44697
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:15.978446 21135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:15.978797 21135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:15.978950 21135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:15.988941 21135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:15.989068 21135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:15.989236 21135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:14:16.003132 21135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:16.004868 21135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:16.006230 21135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:16.019037 21150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:16.019097 21148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:16.019066 21146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:16.020061 21135 server_base.cc:1034] running on GCE node
I20260812 07:14:16.020653 21135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:16.021507 21135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:16.022704 21135 hybrid_clock.cc:648] HybridClock initialized: now 1786518856022660 us; error 32 us; skew 500 ppm
I20260812 07:14:16.022989 21135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:16 dist-test-slave-lgr5 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:16.025996 21135 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:14:16.027311 21135 webserver.cc:460] Webserver started at http://127.21.181.65:34433/ using document root <none> and password file <none>
I20260812 07:14:16.027966 21135 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:16.028070 21135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:16.028383 21135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:16.031060 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "6e3c4f18e7de4c6baa4894da65a89917"
format_stamp: "Formatted at 2026-08-12 07:14:16 on dist-test-slave-lgr5"
server_key: "75be8cb7451cdb52d75e5def3124b1e6"
server_key_iv: "f53e53f51528ca7da86025784b7eb97a"
server_key_version: "encryptionkey@0"
I20260812 07:14:16.031759 21135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "6e3c4f18e7de4c6baa4894da65a89917"
format_stamp: "Formatted at 2026-08-12 07:14:16 on dist-test-slave-lgr5"
server_key: "75be8cb7451cdb52d75e5def3124b1e6"
server_key_iv: "f53e53f51528ca7da86025784b7eb97a"
server_key_version: "encryptionkey@0"
I20260812 07:14:16.036528 21135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260812 07:14:16.039656 21161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:16.040858 21135 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:14:16.040980 21135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "6e3c4f18e7de4c6baa4894da65a89917"
format_stamp: "Formatted at 2026-08-12 07:14:16 on dist-test-slave-lgr5"
server_key: "75be8cb7451cdb52d75e5def3124b1e6"
server_key_iv: "f53e53f51528ca7da86025784b7eb97a"
server_key_version: "encryptionkey@0"
I20260812 07:14:16.041131 21135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:16.053656 21135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:16.057440 21135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:16.057672 21135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:16.058470 21135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:16.059650 21135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:16.059719 21135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:16.059803 21135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:16.059847 21135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:16.089969 21135 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:39107
I20260812 07:14:16.090011 21362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:39107 every 8 connection(s)
I20260812 07:14:16.091174 21135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:14:16.099119 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 21135
I20260812 07:14:16.099340 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:14:16.100240 22229 external_mini_cluster.cc:1428] Setting key 5f94a69d6f36f178fd7477c51b0e9bcc
Aug 12 07:14:16 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:16.110890 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:16.093530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:53495 (local address 127.21.181.126:44697)
0812 07:14:16.093929 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:16.093934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:16.094751 (+ 817us) server_negotiation.cc:407] Connection header received
0812 07:14:16.096596 (+ 1845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:16.096602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:16.096695 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:16.096797 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:16.098258 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.099029 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.101263 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.101712 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.101825 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:16.105592 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:16.105597 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:16.105600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:16.105633 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:16.108054 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:16.108843 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:16.108847 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:16.108849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:16.108964 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:16.109378 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:16.109380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:16.109382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:16.109630 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:16.109738 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:16.110457 (+ 719us) server_negotiation.cc:299] Negotiation successful
0812 07:14:16.110629 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":299,"thread_start_us":201,"threads_started":1}
I20260812 07:14:16.112061 21368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:16.093924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44697 (local address 127.21.181.65:53495)
0812 07:14:16.094603 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:16.094641 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:16.096330 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:16.097042 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:16.097050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:16.097525 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:16.098092 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:16.098112 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.099230 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.099235 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:16.101089 (+ 1854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:16.101103 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.101934 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.101943 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:16.102177 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:16.102841 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:16.102869 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:16.105377 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:16.108228 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:16.108236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:16.108248 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:16.108745 (+ 497us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:16.109061 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:16.109065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:16.109068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:16.109287 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:16.109744 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:16.109750 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:16.110117 (+ 367us) client_negotiation.cc:769] Sending connection context
0812 07:14:16.110381 (+ 264us) client_negotiation.cc:240] Negotiation successful
0812 07:14:16.110843 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":258,"threads_started":1}
I20260812 07:14:16.113476 21363 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44697
I20260812 07:14:16.113770 21363 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:16.114485 21363 heartbeater.cc:511] Master 127.21.181.126:44697 requested a full tablet report, sending...
I20260812 07:14:16.116709 21016 ts_manager.cc:194] Registered new tserver with Master: 6e3c4f18e7de4c6baa4894da65a89917 (127.21.181.65:39107)
I20260812 07:14:16.118178 21016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:53495
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:14:16.161574 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44697
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:16.477900 21374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:16.478158 21374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:16.478317 21374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:16.485486 21374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:16.485596 21374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:16.485767 21374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:14:16.495111 21374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:16.496217 21374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:16.497499 21374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:16.509891 21384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:16.509928 21388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:16.509951 21394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:16.510929 21374 server_base.cc:1034] running on GCE node
I20260812 07:14:16.511531 21374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:16.512512 21374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:16.513723 21374 hybrid_clock.cc:648] HybridClock initialized: now 1786518856513679 us; error 29 us; skew 500 ppm
I20260812 07:14:16.514007 21374 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:16 dist-test-slave-lgr5 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:16.517005 21374 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:14:16.518455 21374 webserver.cc:460] Webserver started at http://127.21.181.66:45713/ using document root <none> and password file <none>
I20260812 07:14:16.519104 21374 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:16.519235 21374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:16.519554 21374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:16.522538 21374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "7889fe2f47a345beb233b62e29b32279"
format_stamp: "Formatted at 2026-08-12 07:14:16 on dist-test-slave-lgr5"
server_key: "0e7686367c5caacc223f80612101c488"
server_key_iv: "d30ad34ec281c48d5123b16a8ee83564"
server_key_version: "encryptionkey@0"
I20260812 07:14:16.523319 21374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "7889fe2f47a345beb233b62e29b32279"
format_stamp: "Formatted at 2026-08-12 07:14:16 on dist-test-slave-lgr5"
server_key: "0e7686367c5caacc223f80612101c488"
server_key_iv: "d30ad34ec281c48d5123b16a8ee83564"
server_key_version: "encryptionkey@0"
I20260812 07:14:16.528421 21374 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260812 07:14:16.531706 21402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:16.532953 21374 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:14:16.533109 21374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "7889fe2f47a345beb233b62e29b32279"
format_stamp: "Formatted at 2026-08-12 07:14:16 on dist-test-slave-lgr5"
server_key: "0e7686367c5caacc223f80612101c488"
server_key_iv: "d30ad34ec281c48d5123b16a8ee83564"
server_key_version: "encryptionkey@0"
I20260812 07:14:16.533324 21374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:16.565234 21374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:16.568960 21374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:16.569237 21374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:16.570155 21374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:16.571468 21374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:16.571555 21374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:16.571693 21374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:16.571753 21374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:16.603935 21374 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:39801
I20260812 07:14:16.603994 21602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:39801 every 8 connection(s)
I20260812 07:14:16.605223 21374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:14:16.613647 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 21374
I20260812 07:14:16.613827 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:14:16.614426 22229 external_mini_cluster.cc:1428] Setting key 245cac1c567680e60815aa4b0b2beea2
Aug 12 07:14:16 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518856, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:16.624922 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:16.607444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:49265 (local address 127.21.181.126:44697)
0812 07:14:16.607645 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:16.607651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:16.608794 (+ 1143us) server_negotiation.cc:407] Connection header received
0812 07:14:16.610646 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:16.610650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:16.610716 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:16.610792 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:16.612218 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.612971 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.615236 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.615788 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.615912 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:16.619308 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:16.619313 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:16.619318 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:16.619354 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:16.622141 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:16.622774 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:16.622779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:16.622782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:16.622922 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:16.623287 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:16.623290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:16.623293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:16.623621 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:16.623760 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:16.624366 (+ 606us) server_negotiation.cc:299] Negotiation successful
0812 07:14:16.624606 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:14:16.625556 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:16.607903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44697 (local address 127.21.181.66:49265)
0812 07:14:16.608640 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:16.608679 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:16.610387 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:16.610986 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:16.610994 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:16.611500 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:16.612043 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:16.612061 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.613138 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.613143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:16.615095 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:16.615108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:16.616011 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:16.616017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:16.616266 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:16.616914 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:16.616942 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:16.619090 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:16.622262 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:16.622269 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:16.622279 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:16.622656 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:16.623016 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:16.623018 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:16.623021 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:16.623185 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:16.623720 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:16.623725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:16.623989 (+ 264us) client_negotiation.cc:769] Sending connection context
0812 07:14:16.624255 (+ 266us) client_negotiation.cc:240] Negotiation successful
0812 07:14:16.624590 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":321,"threads_started":1}
I20260812 07:14:16.626770 21603 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44697
I20260812 07:14:16.627060 21603 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:16.627839 21603 heartbeater.cc:511] Master 127.21.181.126:44697 requested a full tablet report, sending...
I20260812 07:14:16.629331 21016 ts_manager.cc:194] Registered new tserver with Master: 7889fe2f47a345beb233b62e29b32279 (127.21.181.66:39801)
I20260812 07:14:16.630295 21016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:49265
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:14:16.673076 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44697
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:16.988066 21612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:16.988374 21612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:16.988518 21612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:16.995406 21612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:16.995514 21612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:16.995658 21612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:14:17.004871 21612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:44697
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:17.006294 21612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:17.007893 21612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:17.020254 21623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:17.020294 21624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:17.020325 21626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:17.021065 21612 server_base.cc:1034] running on GCE node
I20260812 07:14:17.021633 21612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:17.022482 21612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:17.023701 21612 hybrid_clock.cc:648] HybridClock initialized: now 1786518857023655 us; error 39 us; skew 500 ppm
I20260812 07:14:17.023962 21612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:17 dist-test-slave-lgr5 krb5kdc[20938](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518857, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:17.026887 21612 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:14:17.028210 21612 webserver.cc:460] Webserver started at http://127.21.181.67:38767/ using document root <none> and password file <none>
I20260812 07:14:17.028858 21612 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:17.028975 21612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:17.029263 21612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:17.031994 21612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "b8775f40f153407799935fdce92dba1a"
format_stamp: "Formatted at 2026-08-12 07:14:17 on dist-test-slave-lgr5"
server_key: "bac0ef5436525a5126f0aec597f5136e"
server_key_iv: "68632880f6dfb879e32b72d5b12c2311"
server_key_version: "encryptionkey@0"
I20260812 07:14:17.032774 21612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "b8775f40f153407799935fdce92dba1a"
format_stamp: "Formatted at 2026-08-12 07:14:17 on dist-test-slave-lgr5"
server_key: "bac0ef5436525a5126f0aec597f5136e"
server_key_iv: "68632880f6dfb879e32b72d5b12c2311"
server_key_version: "encryptionkey@0"
I20260812 07:14:17.039170 21612 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260812 07:14:17.042395 21637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:17.043584 21612 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:14:17.043725 21612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "b8775f40f153407799935fdce92dba1a"
format_stamp: "Formatted at 2026-08-12 07:14:17 on dist-test-slave-lgr5"
server_key: "bac0ef5436525a5126f0aec597f5136e"
server_key_iv: "68632880f6dfb879e32b72d5b12c2311"
server_key_version: "encryptionkey@0"
I20260812 07:14:17.043897 21612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:17.071820 21612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:17.075668 21612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:17.075920 21612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:17.076833 21612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:17.078078 21612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:17.078154 21612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:17.078271 21612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:17.078341 21612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:17.108624 21612 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:35011
I20260812 07:14:17.108667 21838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:35011 every 8 connection(s)
I20260812 07:14:17.109789 21612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:14:17.113646 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 21612
I20260812 07:14:17.113816 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:14:17.114351 22229 external_mini_cluster.cc:1428] Setting key 90eac57e1c78707b0cda84efbddf3944
I20260812 07:14:17.120836 21363 heartbeater.cc:503] Master 127.21.181.126:44697 was elected leader, sending a full tablet report...
Aug 12 07:14:17 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518857, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:17.128482 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.112114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:43449 (local address 127.21.181.126:44697)
0812 07:14:17.112289 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.112294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.113346 (+ 1052us) server_negotiation.cc:407] Connection header received
0812 07:14:17.115474 (+ 2128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.115480 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.115586 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.115689 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:17.117436 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.118215 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.119658 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.120061 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.120198 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.123421 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:17.123425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:17.123429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:17.123459 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:17.125800 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.126472 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.126475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.126478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.126596 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.126934 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.126936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.126938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.127183 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:17.127289 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.127913 (+ 624us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.128153 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:14:17.129477 21846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.112515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:44697 (local address 127.21.181.67:43449)
0812 07:14:17.113205 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:17.113241 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:17.115100 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:17.115957 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:17.115969 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:17.116640 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:17.117267 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.117289 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.118336 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.118340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.119544 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.119556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.120261 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.120266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.120460 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.121086 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:17.121110 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:17.123234 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:17.125928 (+ 2694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:17.125934 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:17.125945 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:17.126374 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:17.126678 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:17.126681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:17.126683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:17.126857 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:17.127270 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:17.127275 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:17.127549 (+ 274us) client_negotiation.cc:769] Sending connection context
0812 07:14:17.127837 (+ 288us) client_negotiation.cc:240] Negotiation successful
0812 07:14:17.128284 (+ 447us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":294,"threads_started":1}
I20260812 07:14:17.130638 21839 heartbeater.cc:347] Connected to a master server at 127.21.181.126:44697
I20260812 07:14:17.130903 21839 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:17.131558 21839 heartbeater.cc:511] Master 127.21.181.126:44697 requested a full tablet report, sending...
I20260812 07:14:17.132833 21016 ts_manager.cc:194] Registered new tserver with Master: b8775f40f153407799935fdce92dba1a (127.21.181.67:35011)
I20260812 07:14:17.133579 21016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:43449
I20260812 07:14:17.141441 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:14:17.157414 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.145723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32914 (local address 127.21.181.126:44697)
0812 07:14:17.145872 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.145876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.146381 (+ 505us) server_negotiation.cc:407] Connection header received
0812 07:14:17.146558 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.146564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.146626 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.146714 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:17.148283 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.148968 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.150228 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.150620 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.150705 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.152933 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:17.152937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:17.152941 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:17.152969 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:17.155342 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.155884 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.155887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.155889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.156007 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.156301 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.156304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.156306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.156579 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:17.156755 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.156990 (+ 235us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.157135 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:14:17.160924 21016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:14:17.167558 21016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:14:17.188894 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.180166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.21.181.67:35011)
0812 07:14:17.181211 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.181219 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.181252 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:14:17.181357 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.181365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.181623 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.181827 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:17.182671 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.184223 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.186141 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.187499 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.187617 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.187828 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.188446 (+ 618us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.188598 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":884,"thread_start_us":719,"threads_started":1}
I20260812 07:14:17.188894 21870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.179416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35011 (local address 127.0.0.1:40534)
0812 07:14:17.180464 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:17.180480 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:17.180583 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:17.182107 (+ 1524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:17.182114 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:17.182149 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:17.182459 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.182471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.184380 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.184385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.185981 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.185998 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.187984 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.187991 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.188210 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.188263 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 07:14:17.188362 (+ 99us) client_negotiation.cc:240] Negotiation successful
0812 07:14:17.188542 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":940,"thread_start_us":833,"threads_started":1}
I20260812 07:14:17.190274 21866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.178957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39107 (local address 127.0.0.1:46248)
0812 07:14:17.179653 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:17.179693 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:17.179892 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:17.183159 (+ 3267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:17.183163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:17.183187 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:17.183677 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.183692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.185716 (+ 2024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.185722 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.187599 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.187613 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.189486 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.189493 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.189654 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.189685 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:14:17.189772 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:14:17.189895 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":127,"thread_start_us":1704,"threads_started":1}
I20260812 07:14:17.190562 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.179308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46248 (local address 127.21.181.65:39107)
0812 07:14:17.182598 (+ 3290us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.182606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.182632 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:14:17.182729 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.182736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.182948 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.183162 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:17.183924 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.185573 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.187791 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.189223 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.189403 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.189623 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.190090 (+ 467us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.190246 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3130,"thread_start_us":2970,"threads_started":1}
I20260812 07:14:17.190642 21873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.179656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:39801 (local address 127.0.0.1:44696)
0812 07:14:17.181926 (+ 2270us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:17.181944 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:17.182054 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:17.183401 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:17.183406 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:17.183431 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:17.183859 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.183871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.185804 (+ 1933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.185810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.188106 (+ 2296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.188121 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.189884 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.189890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.190067 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.190098 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:14:17.190185 (+ 87us) client_negotiation.cc:240] Negotiation successful
0812 07:14:17.190298 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2142,"mutex_wait_us":43,"thread_start_us":2000,"threads_started":1}
I20260812 07:14:17.191203 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.179956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.21.181.66:39801)
0812 07:14:17.182845 (+ 2889us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.182850 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.182872 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:14:17.182961 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.182965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.183122 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.183325 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:17.184099 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.185669 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.188274 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.189677 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.189800 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.189981 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.190832 (+ 851us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.190970 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2758,"thread_start_us":2649,"threads_started":1}
I20260812 07:14:17.193063 21729 tablet_service.cc:1467] Processing CreateTablet for tablet 344bccf413fb4dffb348fc76e3af8e4a (DEFAULT_TABLE table=test-table [id=aa7f28fddb11406da800181a38e88098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:17.194296 21729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344bccf413fb4dffb348fc76e3af8e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:17.194759 21246 tablet_service.cc:1467] Processing CreateTablet for tablet 344bccf413fb4dffb348fc76e3af8e4a (DEFAULT_TABLE table=test-table [id=aa7f28fddb11406da800181a38e88098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:17.194761 21494 tablet_service.cc:1467] Processing CreateTablet for tablet 344bccf413fb4dffb348fc76e3af8e4a (DEFAULT_TABLE table=test-table [id=aa7f28fddb11406da800181a38e88098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:17.196110 21246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344bccf413fb4dffb348fc76e3af8e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:17.196133 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 344bccf413fb4dffb348fc76e3af8e4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:17.205822 21879 tablet_bootstrap.cc:492] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Bootstrap starting.
I20260812 07:14:17.208781 21879 tablet_bootstrap.cc:654] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:17.210069 21879 log.cc:824] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:17.210870 21882 tablet_bootstrap.cc:492] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Bootstrap starting.
I20260812 07:14:17.211486 21881 tablet_bootstrap.cc:492] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Bootstrap starting.
I20260812 07:14:17.213397 21879 tablet_bootstrap.cc:492] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: No bootstrap required, opened a new log
I20260812 07:14:17.213662 21879 ts_tablet_manager.cc:1397] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:14:17.214805 21881 tablet_bootstrap.cc:654] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:17.214867 21882 tablet_bootstrap.cc:654] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:17.216045 21882 log.cc:824] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:17.216045 21881 log.cc:824] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:17.218991 21882 tablet_bootstrap.cc:492] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: No bootstrap required, opened a new log
I20260812 07:14:17.219072 21881 tablet_bootstrap.cc:492] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: No bootstrap required, opened a new log
I20260812 07:14:17.219230 21882 ts_tablet_manager.cc:1397] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260812 07:14:17.219323 21881 ts_tablet_manager.cc:1397] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 07:14:17.223068 21879 raft_consensus.cc:348] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.223483 21879 raft_consensus.cc:374] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:17.223589 21879 raft_consensus.cc:729] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8775f40f153407799935fdce92dba1a, State: Initialized, Role: FOLLOWER
I20260812 07:14:17.224197 21879 consensus_queue.cc:260] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.225244 21839 heartbeater.cc:503] Master 127.21.181.126:44697 was elected leader, sending a full tablet report...
I20260812 07:14:17.225878 21879 ts_tablet_manager.cc:1428] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260812 07:14:17.226699 21882 raft_consensus.cc:348] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.227053 21882 raft_consensus.cc:374] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:17.227154 21882 raft_consensus.cc:729] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e3c4f18e7de4c6baa4894da65a89917, State: Initialized, Role: FOLLOWER
I20260812 07:14:17.227664 21882 consensus_queue.cc:260] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.228909 21881 raft_consensus.cc:348] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.229328 21881 raft_consensus.cc:374] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:17.229435 21881 raft_consensus.cc:729] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7889fe2f47a345beb233b62e29b32279, State: Initialized, Role: FOLLOWER
I20260812 07:14:17.230103 21881 consensus_queue.cc:260] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.230456 21882 ts_tablet_manager.cc:1428] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260812 07:14:17.231406 21603 heartbeater.cc:503] Master 127.21.181.126:44697 was elected leader, sending a full tablet report...
I20260812 07:14:17.231914 21881 ts_tablet_manager.cc:1428] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260812 07:14:17.245266 21894 raft_consensus.cc:482] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:14:17.245494 21894 raft_consensus.cc:504] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.246986 21894 leader_election.cc:288] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e3c4f18e7de4c6baa4894da65a89917 (127.21.181.65:39107), b8775f40f153407799935fdce92dba1a (127.21.181.67:35011)
I20260812 07:14:17.255300 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.247657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:39435 (local address 127.21.181.65:39107)
0812 07:14:17.247867 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.247874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.248329 (+ 455us) server_negotiation.cc:407] Connection header received
0812 07:14:17.248547 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.248553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.248627 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.248718 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:17.249289 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.250340 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.253016 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.254202 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.254343 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.254414 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.254928 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.255066 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:14:17.255297 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.247375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:46815 (local address 127.21.181.67:35011)
0812 07:14:17.247857 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.247864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.248607 (+ 743us) server_negotiation.cc:407] Connection header received
0812 07:14:17.248733 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.248737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.248802 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.248879 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:17.249495 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.250510 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.253016 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.254186 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.254343 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.254414 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.254928 (+ 514us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.255069 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260812 07:14:17.255538 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.247229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39107 (local address 127.21.181.66:39435)
0812 07:14:17.248210 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:17.248239 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:17.248398 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:17.248741 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:17.248745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:17.248764 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:17.249154 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.249167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.250468 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.250472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.252854 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.252869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.254428 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.254435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.254737 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.254804 (+ 67us) client_negotiation.cc:769] Sending connection context
0812 07:14:17.254895 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:14:17.255085 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":111,"thread_start_us":962,"threads_started":1}
I20260812 07:14:17.255563 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.247465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:35011 (local address 127.21.181.66:46815)
0812 07:14:17.248523 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:17.248538 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:17.248628 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:17.248913 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:17.248917 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:17.248938 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:17.249364 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.249374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.250630 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.250638 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.252853 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:17.252869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.254428 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.254435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:17.254743 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.254801 (+ 58us) client_negotiation.cc:769] Sending connection context
0812 07:14:17.254895 (+ 94us) client_negotiation.cc:240] Negotiation successful
0812 07:14:17.255084 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":927,"mutex_wait_us":152,"thread_start_us":397,"threads_started":1}
I20260812 07:14:17.256446 21764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344bccf413fb4dffb348fc76e3af8e4a" candidate_uuid: "7889fe2f47a345beb233b62e29b32279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8775f40f153407799935fdce92dba1a" is_pre_election: true
I20260812 07:14:17.256485 21282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344bccf413fb4dffb348fc76e3af8e4a" candidate_uuid: "7889fe2f47a345beb233b62e29b32279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e3c4f18e7de4c6baa4894da65a89917" is_pre_election: true
I20260812 07:14:17.256927 21764 raft_consensus.cc:2454] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7889fe2f47a345beb233b62e29b32279 in term 0.
I20260812 07:14:17.256980 21282 raft_consensus.cc:2454] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7889fe2f47a345beb233b62e29b32279 in term 0.
I20260812 07:14:17.257592 21408 leader_election.cc:302] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6e3c4f18e7de4c6baa4894da65a89917, 7889fe2f47a345beb233b62e29b32279, b8775f40f153407799935fdce92dba1a; no voters:
I20260812 07:14:17.257983 21894 raft_consensus.cc:2781] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:14:17.258101 21894 raft_consensus.cc:482] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:14:17.258193 21894 raft_consensus.cc:3037] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:17.260306 21894 raft_consensus.cc:504] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.260866 21894 leader_election.cc:288] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [CANDIDATE]: Term 1 election: Requested vote from peers 6e3c4f18e7de4c6baa4894da65a89917 (127.21.181.65:39107), b8775f40f153407799935fdce92dba1a (127.21.181.67:35011)
I20260812 07:14:17.261188 21282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344bccf413fb4dffb348fc76e3af8e4a" candidate_uuid: "7889fe2f47a345beb233b62e29b32279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e3c4f18e7de4c6baa4894da65a89917"
I20260812 07:14:17.261303 21764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "344bccf413fb4dffb348fc76e3af8e4a" candidate_uuid: "7889fe2f47a345beb233b62e29b32279" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8775f40f153407799935fdce92dba1a"
I20260812 07:14:17.261334 21282 raft_consensus.cc:3037] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:17.261477 21764 raft_consensus.cc:3037] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:17.263528 21282 raft_consensus.cc:2454] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7889fe2f47a345beb233b62e29b32279 in term 1.
I20260812 07:14:17.263646 21764 raft_consensus.cc:2454] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7889fe2f47a345beb233b62e29b32279 in term 1.
I20260812 07:14:17.263886 21407 leader_election.cc:302] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e3c4f18e7de4c6baa4894da65a89917, 7889fe2f47a345beb233b62e29b32279; no voters:
I20260812 07:14:17.264225 21894 raft_consensus.cc:2781] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:17.264642 21894 raft_consensus.cc:686] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 1 LEADER]: Becoming Leader. State: Replica: 7889fe2f47a345beb233b62e29b32279, State: Running, Role: LEADER
I20260812 07:14:17.265066 21894 consensus_queue.cc:237] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } }
I20260812 07:14:17.269084 21016 catalog_manager.cc:5495] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7889fe2f47a345beb233b62e29b32279 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "7889fe2f47a345beb233b62e29b32279" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7889fe2f47a345beb233b62e29b32279" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 39801 } health_report { overall_health: HEALTHY } } }
I20260812 07:14:17.287528 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.282291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.21.181.66:39801)
0812 07:14:17.282476 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.282482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.282605 (+ 123us) server_negotiation.cc:407] Connection header received
0812 07:14:17.282798 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.282802 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.282872 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.282992 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:14:17.283360 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.284064 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.285555 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.285955 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.286061 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.286516 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:14:17.286956 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:14:17.287092 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.287190 (+ 98us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.287265 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260812 07:14:17.294855 21764 raft_consensus.cc:1261] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 1 FOLLOWER]: Refusing update from remote peer 7889fe2f47a345beb233b62e29b32279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:17.294860 21282 raft_consensus.cc:1261] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 1 FOLLOWER]: Refusing update from remote peer 7889fe2f47a345beb233b62e29b32279: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:17.295889 21901 consensus_queue.cc:1035] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e3c4f18e7de4c6baa4894da65a89917" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:17.296244 21894 consensus_queue.cc:1035] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8775f40f153407799935fdce92dba1a" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 35011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:17.305720 21905 mvcc.cc:205] Tried to move back new op lower bound from 7317581239470673920 to 7317581239361232896. Current Snapshot: MvccSnapshot[applied={T|T < 7317581239470673920}]
I20260812 07:14:17.323514 21016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:17.323745 21016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:17.327405 21016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d147a88c35e45a885ec47d0128c98f0: Sending DeleteTablet for 3 replicas of tablet 344bccf413fb4dffb348fc76e3af8e4a
I20260812 07:14:17.328472 21246 tablet_service.cc:1514] Processing DeleteTablet for tablet 344bccf413fb4dffb348fc76e3af8e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:17 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:46248
I20260812 07:14:17.328603 21729 tablet_service.cc:1514] Processing DeleteTablet for tablet 344bccf413fb4dffb348fc76e3af8e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:17 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:40534
I20260812 07:14:17.328744 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet 344bccf413fb4dffb348fc76e3af8e4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:17 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:44696
I20260812 07:14:17.334304 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 21135
I20260812 07:14:17.346009 21922 tablet_replica.cc:330] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: stopping tablet replica
I20260812 07:14:17.346439 21922 raft_consensus.cc:2229] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:17.346994 21922 raft_consensus.cc:2258] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:17.348320 21923 tablet_replica.cc:330] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: stopping tablet replica
I20260812 07:14:17.348718 21923 raft_consensus.cc:2229] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:17.348951 21922 ts_tablet_manager.cc:1905] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:17.349174 21923 raft_consensus.cc:2258] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:17.352406 21924 tablet_replica.cc:330] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: stopping tablet replica
I20260812 07:14:17.352849 21924 raft_consensus.cc:2229] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:14:17.353528 21924 raft_consensus.cc:2258] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:17.354776 21922 ts_tablet_manager.cc:1918] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 12 07:14:17 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:14:17.354950 21922 log.cc:1196] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/344bccf413fb4dffb348fc76e3af8e4a
I20260812 07:14:17.355547 21922 ts_tablet_manager.cc:1939] T 344bccf413fb4dffb348fc76e3af8e4a P 6e3c4f18e7de4c6baa4894da65a89917: Deleting consensus metadata
I20260812 07:14:17.355907 21924 ts_tablet_manager.cc:1905] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:17.356164 21923 ts_tablet_manager.cc:1905] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:17.357281 20996 catalog_manager.cc:4841] TS 6e3c4f18e7de4c6baa4894da65a89917 (127.21.181.65:39107): tablet 344bccf413fb4dffb348fc76e3af8e4a (table test-table [id=aa7f28fddb11406da800181a38e88098]) successfully deleted
I20260812 07:14:17.362939 21923 ts_tablet_manager.cc:1918] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:17.363094 21923 log.cc:1196] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/344bccf413fb4dffb348fc76e3af8e4a
I20260812 07:14:17.363862 21924 ts_tablet_manager.cc:1918] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:17.364114 21924 log.cc:1196] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/344bccf413fb4dffb348fc76e3af8e4a
I20260812 07:14:17.364696 21923 ts_tablet_manager.cc:1939] T 344bccf413fb4dffb348fc76e3af8e4a P b8775f40f153407799935fdce92dba1a: Deleting consensus metadata
I20260812 07:14:17.364724 21924 ts_tablet_manager.cc:1939] T 344bccf413fb4dffb348fc76e3af8e4a P 7889fe2f47a345beb233b62e29b32279: Deleting consensus metadata
I20260812 07:14:17.365883 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.335245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46260 (local address 127.21.181.65:39107)
0812 07:14:17.335429 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.335436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.337835 (+ 2399us) server_negotiation.cc:407] Connection header received
0812 07:14:17.338741 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.338747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.338857 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.338938 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:17.342686 (+ 3748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.343913 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.349553 (+ 5640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.350224 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.350341 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.358768 (+ 8427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:17.358782 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:17.358798 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:17.358843 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:17.362386 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.363243 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.363252 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.363256 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.363420 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.363905 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.363910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.363913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.364426 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:17.364714 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.365291 (+ 577us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.365524 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:14:17.366459 20997 catalog_manager.cc:4841] TS b8775f40f153407799935fdce92dba1a (127.21.181.67:35011): tablet 344bccf413fb4dffb348fc76e3af8e4a (table test-table [id=aa7f28fddb11406da800181a38e88098]) successfully deleted
I20260812 07:14:17.366649 20994 catalog_manager.cc:4841] TS 7889fe2f47a345beb233b62e29b32279 (127.21.181.66:39801): tablet 344bccf413fb4dffb348fc76e3af8e4a (table test-table [id=aa7f28fddb11406da800181a38e88098]) successfully deleted
I20260812 07:14:17.367156 21357 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:17.455224 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 21135
I20260812 07:14:17.478690 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 21374
Aug 12 07:14:17 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:14:17.494808 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.479419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.21.181.66:39801)
0812 07:14:17.479605 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.479611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.479749 (+ 138us) server_negotiation.cc:407] Connection header received
0812 07:14:17.479944 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.479948 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.480024 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.480142 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:17.481425 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.482231 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.483518 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.483942 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.484048 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.488215 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:17.488233 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:17.488254 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:17.488308 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:17.492065 (+ 3757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.492710 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.492720 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.492724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.492903 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.493221 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.493226 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.493233 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.493690 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:17.494021 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.494185 (+ 164us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.494431 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:17.495568 21592 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:17.604848 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 21374
I20260812 07:14:17.627882 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 21612
Aug 12 07:14:17 dist-test-slave-lgr5 krb5kdc[20938](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518854, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:14:17.641842 21878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:17.628715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.21.181.67:35011)
0812 07:14:17.628874 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:17.628878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:17.628896 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:14:17.629020 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:17.629024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:17.629096 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:17.629180 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:17.630535 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.631237 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.632591 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:17.632972 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:17.633061 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:17.637166 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:17.637180 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:17.637196 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:17.637227 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:17.639648 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.640265 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.640272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.640274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.640423 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:17.640693 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:17.640695 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:17.640697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:17.641024 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:17.641239 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:17.641436 (+ 197us) server_negotiation.cc:299] Negotiation successful
0812 07:14:17.641610 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:14:17.642429 21832 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:17.728222 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 21612
I20260812 07:14:17.750612 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 20960
I20260812 07:14:17.751732 21077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:17.858326 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 20960
2026-08-12T07:14:17Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4361 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-12T07:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:17Z Disabled control of system clock
I20260812 07:14:17.946305 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:36405
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:36405
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:14:18.264895 21941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:18.265147 21941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:18.265232 21941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:18.272051 21941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:18.272176 21941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:18.272223 21941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:18.272279 21941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:18.272320 21941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:18.280926 21941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:36405
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:36405
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:18.281978 21941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:18.283617 21941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:18.294566 21950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:18.294595 21951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:18.295159 21953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:18.295670 21941 server_base.cc:1034] running on GCE node
I20260812 07:14:18.296418 21941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:18.297632 21941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:18.298851 21941 hybrid_clock.cc:648] HybridClock initialized: now 1786518858298802 us; error 36 us; skew 500 ppm
I20260812 07:14:18.299180 21941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:18.301340 21941 webserver.cc:460] Webserver started at http://127.21.181.126:36017/ using document root <none> and password file <none>
I20260812 07:14:18.302001 21941 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:18.302095 21941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:18.302409 21941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:18.305262 21941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "2bb01e6f67a44543bdd04167ce5853fa"
format_stamp: "Formatted at 2026-08-12 07:14:18 on dist-test-slave-lgr5"
server_key: "92ba91f22eecce0581a06816829fae67"
server_key_iv: "151640fe3b8c172de2cf7ae10227f215"
server_key_version: "encryptionkey@0"
I20260812 07:14:18.306078 21941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "2bb01e6f67a44543bdd04167ce5853fa"
format_stamp: "Formatted at 2026-08-12 07:14:18 on dist-test-slave-lgr5"
server_key: "92ba91f22eecce0581a06816829fae67"
server_key_iv: "151640fe3b8c172de2cf7ae10227f215"
server_key_version: "encryptionkey@0"
I20260812 07:14:18.310886 21941 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260812 07:14:18.314132 21964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:18.315316 21941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:14:18.315475 21941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "2bb01e6f67a44543bdd04167ce5853fa"
format_stamp: "Formatted at 2026-08-12 07:14:18 on dist-test-slave-lgr5"
server_key: "92ba91f22eecce0581a06816829fae67"
server_key_iv: "151640fe3b8c172de2cf7ae10227f215"
server_key_version: "encryptionkey@0"
I20260812 07:14:18.315691 21941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:18.333467 21941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:18.334147 21941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:18.334395 21941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:18.351064 21941 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:36405
I20260812 07:14:18.351099 22052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:36405 every 8 connection(s)
I20260812 07:14:18.352260 21941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:18.355719 22053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:18.356465 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 21941
I20260812 07:14:18.356671 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:18.357180 22229 external_mini_cluster.cc:1428] Setting key b890bbd804c6e42fab8a423ca8b5844d
I20260812 07:14:18.368772 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa: Bootstrap starting.
I20260812 07:14:18.370232 22065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:18.359061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38018 (local address 127.21.181.126:36405)
0812 07:14:18.360047 (+ 986us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:18.360057 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:18.360131 (+ 74us) server_negotiation.cc:407] Connection header received
0812 07:14:18.361754 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:18.361774 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:18.362224 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:18.362530 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:18.363073 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:18.364355 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:18.366221 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:18.366866 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:18.366985 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:18.368174 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:18.368182 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:18.368194 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:18.368665 (+ 471us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:18.368737 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:18.368761 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:18.368845 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:18.369072 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 07:14:18.369361 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":353,"threads_started":1}
I20260812 07:14:18.371784 22053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:18.372792 22053 log.cc:824] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:18.375144 22053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa: No bootstrap required, opened a new log
I20260812 07:14:18.381839 22053 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb01e6f67a44543bdd04167ce5853fa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36405 } }
I20260812 07:14:18.382150 22053 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:18.382246 22053 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bb01e6f67a44543bdd04167ce5853fa, State: Initialized, Role: FOLLOWER
I20260812 07:14:18.382740 22053 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb01e6f67a44543bdd04167ce5853fa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36405 } }
I20260812 07:14:18.382943 22053 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:18.383077 22053 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:18.383217 22053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:18.385233 22053 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb01e6f67a44543bdd04167ce5853fa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36405 } }
I20260812 07:14:18.385717 22053 leader_election.cc:302] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bb01e6f67a44543bdd04167ce5853fa; no voters:
I20260812 07:14:18.386245 22053 leader_election.cc:288] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:18.386369 22069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:18.387059 22069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [term 1 LEADER]: Becoming Leader. State: Replica: 2bb01e6f67a44543bdd04167ce5853fa, State: Running, Role: LEADER
I20260812 07:14:18.387532 22069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb01e6f67a44543bdd04167ce5853fa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36405 } }
I20260812 07:14:18.387967 22053 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:18.390095 22072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bb01e6f67a44543bdd04167ce5853fa. Latest consensus state: current_term: 1 leader_uuid: "2bb01e6f67a44543bdd04167ce5853fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb01e6f67a44543bdd04167ce5853fa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36405 } } }
I20260812 07:14:18.390475 22072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:18.391044 22071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bb01e6f67a44543bdd04167ce5853fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bb01e6f67a44543bdd04167ce5853fa" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 36405 } } }
I20260812 07:14:18.391286 22071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:18.392882 22084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:18.397408 22084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:18.405714 22084 catalog_manager.cc:1347] Generated new cluster ID: 37460f5e6d2744b49bde4b1d4f14bcf1
I20260812 07:14:18.405830 22084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:18.417435 22084 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:18.418882 22084 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:18.429570 22084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa: Generated new TSK 0
I20260812 07:14:18.430230 22084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:18.442101 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:36405
--builtin_ntp_servers=127.21.181.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:14:18.796360 22103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:18.796610 22103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:18.796698 22103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:18.803562 22103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:18.803643 22103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:18.803751 22103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:14:18.812398 22103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:18.813457 22103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:18.815174 22103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:18.827404 22124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:18.827442 22109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:18.827525 22112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:18.828240 22103 server_base.cc:1034] running on GCE node
I20260812 07:14:18.828824 22103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:18.829633 22103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:18.830827 22103 hybrid_clock.cc:648] HybridClock initialized: now 1786518858830774 us; error 40 us; skew 500 ppm
I20260812 07:14:18.831092 22103 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:18.833088 22103 webserver.cc:460] Webserver started at http://127.21.181.65:42997/ using document root <none> and password file <none>
I20260812 07:14:18.833683 22103 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:18.833766 22103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:18.834010 22103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:18.836705 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "14ec06f1e1174f4caed0d261968ea244"
format_stamp: "Formatted at 2026-08-12 07:14:18 on dist-test-slave-lgr5"
server_key: "88d6f5dfbfd6995b57c7dad8a40c8812"
server_key_iv: "595f334a387d0128e51e56db7e05e064"
server_key_version: "encryptionkey@0"
I20260812 07:14:18.837409 22103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "14ec06f1e1174f4caed0d261968ea244"
format_stamp: "Formatted at 2026-08-12 07:14:18 on dist-test-slave-lgr5"
server_key: "88d6f5dfbfd6995b57c7dad8a40c8812"
server_key_iv: "595f334a387d0128e51e56db7e05e064"
server_key_version: "encryptionkey@0"
I20260812 07:14:18.842094 22103 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:18.845276 22133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:18.846397 22103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:14:18.846560 22103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "14ec06f1e1174f4caed0d261968ea244"
format_stamp: "Formatted at 2026-08-12 07:14:18 on dist-test-slave-lgr5"
server_key: "88d6f5dfbfd6995b57c7dad8a40c8812"
server_key_iv: "595f334a387d0128e51e56db7e05e064"
server_key_version: "encryptionkey@0"
I20260812 07:14:18.846745 22103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:18.863857 22103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:18.864629 22103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:18.864904 22103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:18.865808 22103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:18.867118 22103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:18.867209 22103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:18.867308 22103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:18.867389 22103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:18.897354 22103 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:39451
I20260812 07:14:18.897418 22359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:39451 every 8 connection(s)
I20260812 07:14:18.898692 22103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:14:18.905786 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22103
I20260812 07:14:18.905938 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:14:18.906328 22229 external_mini_cluster.cc:1428] Setting key a2fcdff595fcb3717dedf0f28e26a238
I20260812 07:14:18.909124 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:36405
--builtin_ntp_servers=127.21.181.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:14:18.911206 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:18.901066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:40783 (local address 127.21.181.126:36405)
0812 07:14:18.901479 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:18.901485 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:18.902622 (+ 1137us) server_negotiation.cc:407] Connection header received
0812 07:14:18.904371 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:18.904377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:18.904459 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:18.904556 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:18.905601 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:18.906491 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:18.908546 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:18.908940 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:18.909063 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:18.910376 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:18.910382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:18.910385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:18.910402 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:18.910440 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:18.910451 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:18.910526 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:18.910739 (+ 213us) server_negotiation.cc:299] Negotiation successful
0812 07:14:18.910861 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":204,"threads_started":1}
I20260812 07:14:18.912119 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:18.901624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:36405 (local address 127.21.181.65:40783)
0812 07:14:18.902462 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:18.902507 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:18.904119 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:18.904809 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:18.904817 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:18.904898 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:18.905432 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:18.905449 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:18.906625 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:18.906629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:18.908412 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:18.908426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:18.909163 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:18.909170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:18.909433 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:18.910079 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:18.910082 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:18.910118 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:18.910194 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:18.910540 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:18.910552 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:18.910560 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:14:18.910716 (+ 156us) client_negotiation.cc:240] Negotiation successful
0812 07:14:18.911083 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":265,"threads_started":1}
I20260812 07:14:18.913522 22362 heartbeater.cc:347] Connected to a master server at 127.21.181.126:36405
I20260812 07:14:18.913808 22362 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:18.914633 22362 heartbeater.cc:511] Master 127.21.181.126:36405 requested a full tablet report, sending...
I20260812 07:14:18.916860 22002 ts_manager.cc:194] Registered new tserver with Master: 14ec06f1e1174f4caed0d261968ea244 (127.21.181.65:39451)
I20260812 07:14:18.918709 22002 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.65:40783
W20260812 07:14:19.233250 22372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:19.233513 22372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:19.233618 22372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:19.240527 22372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:19.240610 22372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:19.240715 22372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:14:19.249199 22372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:19.250305 22372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:19.251993 22372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:19.264048 22384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:19.264278 22383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:19.264120 22387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:19.265097 22372 server_base.cc:1034] running on GCE node
I20260812 07:14:19.265679 22372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:19.266556 22372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:19.267751 22372 hybrid_clock.cc:648] HybridClock initialized: now 1786518859267699 us; error 45 us; skew 500 ppm
I20260812 07:14:19.268056 22372 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:19.270134 22372 webserver.cc:460] Webserver started at http://127.21.181.66:40983/ using document root <none> and password file <none>
I20260812 07:14:19.270771 22372 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:19.270867 22372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:19.271159 22372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:19.273828 22372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "66167786f28b4ba3aacb5b531d068028"
format_stamp: "Formatted at 2026-08-12 07:14:19 on dist-test-slave-lgr5"
server_key: "c6130ce20d6a2f03b799a0d55dc6ce82"
server_key_iv: "7557466bd4feb0a03364c7d1414af690"
server_key_version: "encryptionkey@0"
I20260812 07:14:19.274577 22372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "66167786f28b4ba3aacb5b531d068028"
format_stamp: "Formatted at 2026-08-12 07:14:19 on dist-test-slave-lgr5"
server_key: "c6130ce20d6a2f03b799a0d55dc6ce82"
server_key_iv: "7557466bd4feb0a03364c7d1414af690"
server_key_version: "encryptionkey@0"
I20260812 07:14:19.279290 22372 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:14:19.282461 22397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:19.283638 22372 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:14:19.283784 22372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "66167786f28b4ba3aacb5b531d068028"
format_stamp: "Formatted at 2026-08-12 07:14:19 on dist-test-slave-lgr5"
server_key: "c6130ce20d6a2f03b799a0d55dc6ce82"
server_key_iv: "7557466bd4feb0a03364c7d1414af690"
server_key_version: "encryptionkey@0"
I20260812 07:14:19.283967 22372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:19.298854 22372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:19.299506 22372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:19.299739 22372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:19.300607 22372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:19.301878 22372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:19.301962 22372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:19.302053 22372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:19.302106 22372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:19.329628 22372 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:38911
I20260812 07:14:19.329700 22619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:38911 every 8 connection(s)
I20260812 07:14:19.330777 22372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:14:19.337427 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22372
I20260812 07:14:19.337603 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:14:19.338145 22229 external_mini_cluster.cc:1428] Setting key ec3926c8274005299db38aff77ece4a8
I20260812 07:14:19.341279 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:36405
--builtin_ntp_servers=127.21.181.84:33009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:14:19.342999 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.333237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:40423 (local address 127.21.181.126:36405)
0812 07:14:19.333397 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:19.333401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:19.334735 (+ 1334us) server_negotiation.cc:407] Connection header received
0812 07:14:19.336470 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:19.336475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:19.336558 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:19.336646 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:19.337693 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.338471 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.340169 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.340814 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.340966 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.342172 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:19.342180 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:19.342185 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:19.342210 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:19.342252 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:19.342270 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:19.342374 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:19.342551 (+ 177us) server_negotiation.cc:299] Negotiation successful
0812 07:14:19.342660 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:14:19.343735 22625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.333712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:36405 (local address 127.21.181.66:40423)
0812 07:14:19.334578 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:19.334624 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:19.336215 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:19.336884 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:19.336892 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:19.336972 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:19.337525 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.337543 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.338617 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.338621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.340003 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.340020 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.341028 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.341033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.341227 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.341865 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:19.341867 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:19.341902 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:19.341972 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:19.342373 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:19.342378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:19.342383 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:14:19.342514 (+ 131us) client_negotiation.cc:240] Negotiation successful
0812 07:14:19.342774 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":289,"threads_started":1}
I20260812 07:14:19.345091 22621 heartbeater.cc:347] Connected to a master server at 127.21.181.126:36405
I20260812 07:14:19.345417 22621 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:19.346284 22621 heartbeater.cc:511] Master 127.21.181.126:36405 requested a full tablet report, sending...
I20260812 07:14:19.347694 22002 ts_manager.cc:194] Registered new tserver with Master: 66167786f28b4ba3aacb5b531d068028 (127.21.181.66:38911)
I20260812 07:14:19.348508 22002 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.66:40423
W20260812 07:14:19.663769 22627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:19.664014 22627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:19.664134 22627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:19.670990 22627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:19.671074 22627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:19.671205 22627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:14:19.679822 22627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:33009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:36405
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:19.680986 22627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:19.682652 22627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:19.700112 22638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:19.700816 22639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:19.702214 22644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:19.703387 22627 server_base.cc:1034] running on GCE node
I20260812 07:14:19.704142 22627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:19.705330 22627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:19.706578 22627 hybrid_clock.cc:648] HybridClock initialized: now 1786518859706490 us; error 75 us; skew 500 ppm
I20260812 07:14:19.706972 22627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:19.710119 22627 webserver.cc:460] Webserver started at http://127.21.181.67:43161/ using document root <none> and password file <none>
I20260812 07:14:19.710924 22627 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:19.711053 22627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:19.711385 22627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:19.715358 22627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "5c62028dc9a6484e82b9a463600938fc"
format_stamp: "Formatted at 2026-08-12 07:14:19 on dist-test-slave-lgr5"
server_key: "1ebafa761f460a289b34029d397fc5a6"
server_key_iv: "71938940a911f844a9ad1d60548fb249"
server_key_version: "encryptionkey@0"
I20260812 07:14:19.716383 22627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "5c62028dc9a6484e82b9a463600938fc"
format_stamp: "Formatted at 2026-08-12 07:14:19 on dist-test-slave-lgr5"
server_key: "1ebafa761f460a289b34029d397fc5a6"
server_key_iv: "71938940a911f844a9ad1d60548fb249"
server_key_version: "encryptionkey@0"
I20260812 07:14:19.722465 22627 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260812 07:14:19.727032 22654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:19.728621 22627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260812 07:14:19.728786 22627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "5c62028dc9a6484e82b9a463600938fc"
format_stamp: "Formatted at 2026-08-12 07:14:19 on dist-test-slave-lgr5"
server_key: "1ebafa761f460a289b34029d397fc5a6"
server_key_iv: "71938940a911f844a9ad1d60548fb249"
server_key_version: "encryptionkey@0"
I20260812 07:14:19.729010 22627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:19.751617 22627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:19.752355 22627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:19.752617 22627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:19.753469 22627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:19.754812 22627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:19.754913 22627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:19.755009 22627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:19.755089 22627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:19.784058 22627 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38773
I20260812 07:14:19.784163 22863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38773 every 8 connection(s)
I20260812 07:14:19.785264 22627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:14:19.791254 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22627
I20260812 07:14:19.791414 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:14:19.791923 22229 external_mini_cluster.cc:1428] Setting key 3490d05c356c2002b11e28b71355ef8c
I20260812 07:14:19.797397 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.787603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:46391 (local address 127.21.181.126:36405)
0812 07:14:19.787761 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:19.787765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:19.789257 (+ 1492us) server_negotiation.cc:407] Connection header received
0812 07:14:19.790962 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:19.790968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:19.791049 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:19.791136 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:19.792373 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.793329 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.795209 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.795608 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.795699 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.796812 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:19.796816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:19.796820 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:19.796836 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:19.796868 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:19.796880 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:19.796942 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:19.797101 (+ 159us) server_negotiation.cc:299] Negotiation successful
0812 07:14:19.797175 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:14:19.798231 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.788239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:36405 (local address 127.21.181.67:46391)
0812 07:14:19.789105 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:19.789147 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:19.790705 (+ 1558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:19.791361 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:19.791372 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:19.791474 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:19.792127 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.792153 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.793485 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.793489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.795086 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.795098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.795768 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.795773 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.795974 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.796541 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:19.796543 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:19.796579 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:19.796649 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:19.796947 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:19.796952 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:19.796957 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:14:19.797074 (+ 117us) client_negotiation.cc:240] Negotiation successful
0812 07:14:19.797337 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":288,"threads_started":1}
I20260812 07:14:19.799337 22864 heartbeater.cc:347] Connected to a master server at 127.21.181.126:36405
I20260812 07:14:19.799592 22864 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:19.800283 22864 heartbeater.cc:511] Master 127.21.181.126:36405 requested a full tablet report, sending...
I20260812 07:14:19.801522 22002 ts_manager.cc:194] Registered new tserver with Master: 5c62028dc9a6484e82b9a463600938fc (127.21.181.67:38773)
I20260812 07:14:19.802276 22002 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.67:46391
I20260812 07:14:19.806605 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:14:19.815327 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.810307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42664 (local address 127.21.181.126:36405)
0812 07:14:19.810455 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:19.810459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:19.810904 (+ 445us) server_negotiation.cc:407] Connection header received
0812 07:14:19.811024 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:19.811028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:19.811089 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:19.811160 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:19.811553 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.812240 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.813563 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.813940 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.814028 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.814875 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:19.814880 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:19.814883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:19.814897 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:19.814925 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:19.814936 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:19.815011 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:19.815071 (+ 60us) server_negotiation.cc:299] Negotiation successful
0812 07:14:19.815135 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:14:19.818483 22002 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42664:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:14:19.824954 22002 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:14:19.859040 22890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.835726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43456)
0812 07:14:19.847163 (+ 11437us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:19.847214 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:19.847369 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:19.851764 (+ 4395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:19.851769 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:19.851793 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:19.852704 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.852720 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.854553 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.854557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.856560 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.856573 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.858185 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.858189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.858346 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.858380 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:14:19.858500 (+ 120us) client_negotiation.cc:240] Negotiation successful
0812 07:14:19.858680 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11187,"mutex_wait_us":1,"thread_start_us":15449,"threads_started":1}
I20260812 07:14:19.859064 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.836006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43456 (local address 127.21.181.65:39451)
0812 07:14:19.851110 (+ 15104us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:19.851126 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:19.851156 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:14:19.851253 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:19.851261 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:19.851545 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:19.852907 (+ 1362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:19.853130 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.854428 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.856741 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.858023 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.858163 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.858415 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:19.858578 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 07:14:19.858737 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14930,"thread_start_us":14833,"threads_started":1}
I20260812 07:14:19.859714 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.837017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:38911 (local address 127.0.0.1:54994)
0812 07:14:19.851444 (+ 14427us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:19.851461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:19.851596 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:19.852565 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:19.852569 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:19.852592 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:19.852933 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.852942 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.854147 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.854153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.856864 (+ 2711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.856875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.858933 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.858939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.859133 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.859167 (+ 34us) client_negotiation.cc:769] Sending connection context
0812 07:14:19.859252 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:14:19.859364 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14309,"mutex_wait_us":140,"thread_start_us":13152,"threads_started":1}
I20260812 07:14:19.861349 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.836147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56324 (local address 127.21.181.67:38773)
0812 07:14:19.849447 (+ 13300us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:19.849455 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:19.850619 (+ 1164us) server_negotiation.cc:407] Connection header received
0812 07:14:19.850790 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:19.850798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:19.851045 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:19.852021 (+ 976us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:19.852446 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.854086 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.858513 (+ 4427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.860151 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.860272 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.860522 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:19.860980 (+ 458us) server_negotiation.cc:299] Negotiation successful
0812 07:14:19.861104 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13134,"thread_start_us":13314,"threads_started":1}
I20260812 07:14:19.861370 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.835885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38773 (local address 127.0.0.1:56324)
0812 07:14:19.850506 (+ 14621us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:19.850519 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:19.850633 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:19.851288 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:19.851295 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:19.851330 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:19.851791 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.851803 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.854213 (+ 2410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.854217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.856853 (+ 2636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:19.856869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.860645 (+ 3776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.860652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:19.860809 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.860839 (+ 30us) client_negotiation.cc:769] Sending connection context
0812 07:14:19.860932 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:14:19.861055 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14515,"mutex_wait_us":162,"thread_start_us":10897,"threads_started":1}
I20260812 07:14:19.861770 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:19.837684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.21.181.66:38911)
0812 07:14:19.850008 (+ 12324us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:19.850017 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:19.851570 (+ 1553us) server_negotiation.cc:407] Connection header received
0812 07:14:19.851819 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:19.851827 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:19.852032 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:19.852227 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:19.853090 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.854016 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.857069 (+ 3053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:19.858447 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:19.858584 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:19.858805 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:19.860116 (+ 1311us) server_negotiation.cc:299] Negotiation successful
0812 07:14:19.860283 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12174,"thread_start_us":12064,"threads_started":1}
I20260812 07:14:19.863308 22243 tablet_service.cc:1467] Processing CreateTablet for tablet 654dfbc332114b80bda040520724b0f9 (DEFAULT_TABLE table=test-table [id=bde27006b20048aeac183ce19607e60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:19.864526 22243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654dfbc332114b80bda040520724b0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:19.864799 22736 tablet_service.cc:1467] Processing CreateTablet for tablet 654dfbc332114b80bda040520724b0f9 (DEFAULT_TABLE table=test-table [id=bde27006b20048aeac183ce19607e60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:19.865242 22494 tablet_service.cc:1467] Processing CreateTablet for tablet 654dfbc332114b80bda040520724b0f9 (DEFAULT_TABLE table=test-table [id=bde27006b20048aeac183ce19607e60b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:19.866285 22736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654dfbc332114b80bda040520724b0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:19.866508 22494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 654dfbc332114b80bda040520724b0f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:19.880131 22896 tablet_bootstrap.cc:492] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Bootstrap starting.
I20260812 07:14:19.881912 22898 tablet_bootstrap.cc:492] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Bootstrap starting.
I20260812 07:14:19.883909 22896 tablet_bootstrap.cc:654] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:19.884599 22898 tablet_bootstrap.cc:654] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:19.885247 22896 log.cc:824] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:19.885516 22898 log.cc:824] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:19.886842 22897 tablet_bootstrap.cc:492] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244: Bootstrap starting.
I20260812 07:14:19.889915 22896 tablet_bootstrap.cc:492] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: No bootstrap required, opened a new log
I20260812 07:14:19.890213 22896 ts_tablet_manager.cc:1397] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260812 07:14:19.890691 22897 tablet_bootstrap.cc:654] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:19.892130 22897 log.cc:824] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:19.894862 22898 tablet_bootstrap.cc:492] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: No bootstrap required, opened a new log
I20260812 07:14:19.895124 22897 tablet_bootstrap.cc:492] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244: No bootstrap required, opened a new log
I20260812 07:14:19.895124 22898 ts_tablet_manager.cc:1397] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.004s
I20260812 07:14:19.895375 22897 ts_tablet_manager.cc:1397] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260812 07:14:19.899926 22896 raft_consensus.cc:348] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:19.900372 22896 raft_consensus.cc:374] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:19.900463 22896 raft_consensus.cc:729] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66167786f28b4ba3aacb5b531d068028, State: Initialized, Role: FOLLOWER
I20260812 07:14:19.901041 22896 consensus_queue.cc:260] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:19.905346 22898 raft_consensus.cc:348] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:19.905781 22898 raft_consensus.cc:374] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:19.905877 22898 raft_consensus.cc:729] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c62028dc9a6484e82b9a463600938fc, State: Initialized, Role: FOLLOWER
I20260812 07:14:19.905927 22897 raft_consensus.cc:348] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:19.906308 22897 raft_consensus.cc:374] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:19.906409 22897 raft_consensus.cc:729] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14ec06f1e1174f4caed0d261968ea244, State: Initialized, Role: FOLLOWER
I20260812 07:14:19.906586 22898 consensus_queue.cc:260] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:19.907069 22897 consensus_queue.cc:260] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:19.908747 22621 heartbeater.cc:503] Master 127.21.181.126:36405 was elected leader, sending a full tablet report...
I20260812 07:14:19.909114 22896 ts_tablet_manager.cc:1428] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Time spent starting tablet: real 0.019s user 0.015s sys 0.005s
I20260812 07:14:19.912523 22864 heartbeater.cc:503] Master 127.21.181.126:36405 was elected leader, sending a full tablet report...
I20260812 07:14:19.915470 22898 ts_tablet_manager.cc:1428] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Time spent starting tablet: real 0.020s user 0.016s sys 0.001s
I20260812 07:14:19.915890 22362 heartbeater.cc:503] Master 127.21.181.126:36405 was elected leader, sending a full tablet report...
I20260812 07:14:19.917557 22897 ts_tablet_manager.cc:1428] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244: Time spent starting tablet: real 0.022s user 0.012s sys 0.007s
I20260812 07:14:20.016920 22907 raft_consensus.cc:482] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:14:20.017218 22907 raft_consensus.cc:504] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:20.018947 22907 leader_election.cc:288] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14ec06f1e1174f4caed0d261968ea244 (127.21.181.65:39451), 66167786f28b4ba3aacb5b531d068028 (127.21.181.66:38911)
I20260812 07:14:20.025199 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.019257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:48769 (local address 127.21.181.65:39451)
0812 07:14:20.019457 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:20.019463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:20.019492 (+ 29us) server_negotiation.cc:407] Connection header received
0812 07:14:20.019698 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:20.019704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:20.019797 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:20.019877 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:20.020441 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.021366 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.023189 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.024305 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.024441 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.024505 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:20.024837 (+ 332us) server_negotiation.cc:299] Negotiation successful
0812 07:14:20.024927 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:14:20.025221 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.019160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.21.181.67:48769)
0812 07:14:20.019339 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.019359 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:20.019481 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:20.020097 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:20.020102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:20.020133 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:20.020286 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:20.020295 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.021505 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.021511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:20.023033 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:20.023045 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.024467 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.024474 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:20.024676 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.024712 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:14:20.024825 (+ 113us) client_negotiation.cc:240] Negotiation successful
0812 07:14:20.024961 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":1}
I20260812 07:14:20.025754 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.020030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:56087 (local address 127.21.181.66:38911)
0812 07:14:20.020247 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:20.020253 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:20.020270 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:14:20.020428 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:20.020434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:20.020516 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:20.020582 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:20.021134 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.022011 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.023906 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.024856 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.024980 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.025054 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:20.025398 (+ 344us) server_negotiation.cc:299] Negotiation successful
0812 07:14:20.025526 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260812 07:14:20.026206 22910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.019343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:38911 (local address 127.21.181.67:56087)
0812 07:14:20.020108 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.020127 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:20.020253 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:20.020619 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:20.020624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:20.020650 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:20.020994 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:20.021007 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.022162 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.022166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:20.023722 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:20.023737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.025022 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.025032 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:20.025200 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.025236 (+ 36us) client_negotiation.cc:769] Sending connection context
0812 07:14:20.025912 (+ 676us) client_negotiation.cc:240] Negotiation successful
0812 07:14:20.026017 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":411,"threads_started":1}
I20260812 07:14:20.026171 22270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654dfbc332114b80bda040520724b0f9" candidate_uuid: "5c62028dc9a6484e82b9a463600938fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ec06f1e1174f4caed0d261968ea244" is_pre_election: true
I20260812 07:14:20.026695 22270 raft_consensus.cc:2454] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c62028dc9a6484e82b9a463600938fc in term 0.
I20260812 07:14:20.027166 22535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654dfbc332114b80bda040520724b0f9" candidate_uuid: "5c62028dc9a6484e82b9a463600938fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66167786f28b4ba3aacb5b531d068028" is_pre_election: true
I20260812 07:14:20.027386 22656 leader_election.cc:302] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ec06f1e1174f4caed0d261968ea244, 5c62028dc9a6484e82b9a463600938fc; no voters:
I20260812 07:14:20.027768 22907 raft_consensus.cc:2781] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:14:20.027725 22535 raft_consensus.cc:2454] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c62028dc9a6484e82b9a463600938fc in term 0.
I20260812 07:14:20.027900 22907 raft_consensus.cc:482] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:14:20.028003 22907 raft_consensus.cc:3037] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:20.030193 22907 raft_consensus.cc:504] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
I20260812 07:14:20.030705 22907 leader_election.cc:288] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [CANDIDATE]: Term 1 election: Requested vote from peers 14ec06f1e1174f4caed0d261968ea244 (127.21.181.65:39451), 66167786f28b4ba3aacb5b531d068028 (127.21.181.66:38911)
I20260812 07:14:20.031086 22270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654dfbc332114b80bda040520724b0f9" candidate_uuid: "5c62028dc9a6484e82b9a463600938fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14ec06f1e1174f4caed0d261968ea244"
I20260812 07:14:20.031183 22535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "654dfbc332114b80bda040520724b0f9" candidate_uuid: "5c62028dc9a6484e82b9a463600938fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66167786f28b4ba3aacb5b531d068028"
I20260812 07:14:20.031286 22270 raft_consensus.cc:3037] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:20.031366 22535 raft_consensus.cc:3037] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:20.034464 22270 raft_consensus.cc:2454] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c62028dc9a6484e82b9a463600938fc in term 1.
I20260812 07:14:20.034471 22535 raft_consensus.cc:2454] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c62028dc9a6484e82b9a463600938fc in term 1.
I20260812 07:14:20.034817 22656 leader_election.cc:302] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14ec06f1e1174f4caed0d261968ea244, 5c62028dc9a6484e82b9a463600938fc; no voters:
I20260812 07:14:20.035081 22907 raft_consensus.cc:2781] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:20.035481 22907 raft_consensus.cc:686] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 1 LEADER]: Becoming Leader. State: Replica: 5c62028dc9a6484e82b9a463600938fc, State: Running, Role: LEADER
I20260812 07:14:20.035988 22907 consensus_queue.cc:237] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } }
W20260812 07:14:20.036929 22865 tablet.cc:2369] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:14:20.040732 22001 catalog_manager.cc:5495] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c62028dc9a6484e82b9a463600938fc (127.21.181.67). New cstate: current_term: 1 leader_uuid: "5c62028dc9a6484e82b9a463600938fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c62028dc9a6484e82b9a463600938fc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 } health_report { overall_health: UNKNOWN } } }
W20260812 07:14:20.082723 22623 tablet.cc:2369] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:14:20.094156 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.089363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.21.181.67:38773)
0812 07:14:20.089526 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:20.089530 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:20.089578 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:14:20.089748 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:20.089753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:20.089876 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:20.090023 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:14:20.090307 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.091020 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.092411 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.092806 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.092898 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.093290 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:14:20.093758 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:14:20.093862 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:20.093905 (+ 43us) server_negotiation.cc:299] Negotiation successful
0812 07:14:20.093964 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:14:20.101487 22535 raft_consensus.cc:1261] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 1 FOLLOWER]: Refusing update from remote peer 5c62028dc9a6484e82b9a463600938fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:20.101534 22270 raft_consensus.cc:1261] T 654dfbc332114b80bda040520724b0f9 P 14ec06f1e1174f4caed0d261968ea244 [term 1 FOLLOWER]: Refusing update from remote peer 5c62028dc9a6484e82b9a463600938fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:20.102399 22911 consensus_queue.cc:1035] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "14ec06f1e1174f4caed0d261968ea244" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 39451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:20.102721 22907 consensus_queue.cc:1035] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "66167786f28b4ba3aacb5b531d068028" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38911 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:20.113957 22914 mvcc.cc:205] Tried to move back new op lower bound from 7317581250966675456 to 7317581250711584768. Current Snapshot: MvccSnapshot[applied={T|T < 7317581250966675456}]
I20260812 07:14:20.117470 22917 mvcc.cc:205] Tried to move back new op lower bound from 7317581250966675456 to 7317581250711584768. Current Snapshot: MvccSnapshot[applied={T|T < 7317581250966675456 or (T in {7317581250966675456})}]
I20260812 07:14:20.125161 22002 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:20.125382 22002 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42664:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:20.128615 22002 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2bb01e6f67a44543bdd04167ce5853fa: Sending DeleteTablet for 3 replicas of tablet 654dfbc332114b80bda040520724b0f9
I20260812 07:14:20.129582 22243 tablet_service.cc:1514] Processing DeleteTablet for tablet 654dfbc332114b80bda040520724b0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:20 UTC) from {username='slave'} at 127.0.0.1:43456
I20260812 07:14:20.129688 22494 tablet_service.cc:1514] Processing DeleteTablet for tablet 654dfbc332114b80bda040520724b0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:20 UTC) from {username='slave'} at 127.0.0.1:54994
I20260812 07:14:20.129706 22736 tablet_service.cc:1514] Processing DeleteTablet for tablet 654dfbc332114b80bda040520724b0f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:20 UTC) from {username='slave'} at 127.0.0.1:56324
I20260812 07:14:20.134146 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22103
I20260812 07:14:20.146768 22929 tablet_replica.cc:330] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: stopping tablet replica
I20260812 07:14:20.147186 22929 raft_consensus.cc:2229] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:20.147678 22929 raft_consensus.cc:2258] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:20.148624 22933 tablet_replica.cc:330] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: stopping tablet replica
I20260812 07:14:20.149044 22933 raft_consensus.cc:2229] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:14:20.149696 22933 raft_consensus.cc:2258] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:20.150242 22891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.135450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43462 (local address 127.21.181.65:39451)
0812 07:14:20.135623 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:20.135630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:20.138155 (+ 2525us) server_negotiation.cc:407] Connection header received
0812 07:14:20.138888 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:20.138894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:20.139019 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:20.139091 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:20.140189 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.141368 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.145338 (+ 3970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.146022 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.146117 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.148654 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:20.148666 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:20.148680 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:20.149372 (+ 692us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:20.149431 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:20.149456 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:20.149589 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:20.149761 (+ 172us) server_negotiation.cc:299] Negotiation successful
0812 07:14:20.149876 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:14:20.150964 22350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:20.152666 22929 ts_tablet_manager.cc:1905] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:20.157541 22933 ts_tablet_manager.cc:1905] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:20.166965 22933 ts_tablet_manager.cc:1918] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:20.167145 22933 log.cc:1196] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/654dfbc332114b80bda040520724b0f9
I20260812 07:14:20.167863 22933 ts_tablet_manager.cc:1939] T 654dfbc332114b80bda040520724b0f9 P 5c62028dc9a6484e82b9a463600938fc: Deleting consensus metadata
I20260812 07:14:20.169692 22929 ts_tablet_manager.cc:1918] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:20.169718 21969 catalog_manager.cc:4841] TS 5c62028dc9a6484e82b9a463600938fc (127.21.181.67:38773): tablet 654dfbc332114b80bda040520724b0f9 (table test-table [id=bde27006b20048aeac183ce19607e60b]) successfully deleted
I20260812 07:14:20.169865 22929 log.cc:1196] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/654dfbc332114b80bda040520724b0f9
I20260812 07:14:20.170400 22929 ts_tablet_manager.cc:1939] T 654dfbc332114b80bda040520724b0f9 P 66167786f28b4ba3aacb5b531d068028: Deleting consensus metadata
I20260812 07:14:20.172051 21965 catalog_manager.cc:4841] TS 66167786f28b4ba3aacb5b531d068028 (127.21.181.66:38911): tablet 654dfbc332114b80bda040520724b0f9 (table test-table [id=bde27006b20048aeac183ce19607e60b]) successfully deleted
I20260812 07:14:20.253268 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22103
W20260812 07:14:20.291119 21968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.181.65:39451 (error 108)
I20260812 07:14:20.291834 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22372
I20260812 07:14:20.293573 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.293036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43472)
0812 07:14:20.293268 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.293387 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260812 07:14:20.293787 21968 catalog_manager.cc:4584] TS 14ec06f1e1174f4caed0d261968ea244 (127.21.181.65:39451): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 654dfbc332114b80bda040520724b0f9: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
I20260812 07:14:20.299520 22895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.293006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55002 (local address 127.21.181.66:38911)
0812 07:14:20.293184 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:20.293191 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:20.293211 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:14:20.293279 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:20.293284 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:20.293408 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:20.293522 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:20.294009 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.294806 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.296646 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.297043 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.297138 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.298228 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:20.298237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:20.298251 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:20.298815 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:20.298862 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:20.298882 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:20.298965 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:20.299102 (+ 137us) server_negotiation.cc:299] Negotiation successful
0812 07:14:20.299190 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:20.300257 22612 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:20.347608 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.347057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43480)
0812 07:14:20.347289 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.347399 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260812 07:14:20.348786 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.348385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43494)
0812 07:14:20.348542 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.348630 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260812 07:14:20.409399 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.408863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43504)
0812 07:14:20.409110 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.409217 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260812 07:14:20.410598 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.410173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43520)
0812 07:14:20.410342 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.410433 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260812 07:14:20.414162 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22372
I20260812 07:14:20.434638 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22627
I20260812 07:14:20.447742 22892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.438916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56348 (local address 127.21.181.67:38773)
0812 07:14:20.439102 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:20.439108 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:20.439250 (+ 142us) server_negotiation.cc:407] Connection header received
0812 07:14:20.439428 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:20.439432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:20.439514 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:20.439608 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:20.440031 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.440918 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.442248 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:20.442693 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:20.442810 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:20.446419 (+ 3609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:20.446428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:20.446443 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:20.447038 (+ 595us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:20.447093 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:20.447114 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:20.447216 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:20.447346 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 07:14:20.447440 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:14:20.448423 22859 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:20.491338 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.490875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43536)
0812 07:14:20.491087 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.491176 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:14:20.492403 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.491994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43538)
0812 07:14:20.492218 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.492291 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260812 07:14:20.569486 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22627
I20260812 07:14:20.593600 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 21941
I20260812 07:14:20.594658 22046 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:20.702677 22888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:20.702202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:39451 (local address 127.0.0.1:43550)
0812 07:14:20.702380 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:20.702478 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:39451: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260812 07:14:20.702678 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 21941
2026-08-12T07:14:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2842 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:20 dist-test-slave-lgr5 krb5kdc[22953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:20 dist-test-slave-lgr5 krb5kdc[22953](info): set up 2 sockets
Aug 12 07:14:20 dist-test-slave-lgr5 krb5kdc[22953](info): commencing operation
krb5kdc: starting...
W20260812 07:14:21.954790 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.205s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:22 dist-test-slave-lgr5 krb5kdc[22953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:14:22.208572 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38513
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:38513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:22.524509 22980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:22.524761 22980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:22.524889 22980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:22.531690 22980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:22.531783 22980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:22.531857 22980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:22.531922 22980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:22.531977 22980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:22.541181 22980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:38513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:22.542310 22980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:22.543495 22980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:22.554227 23000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:22.554268 22999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:22.554268 23007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:22.555115 22980 server_base.cc:1034] running on GCE node
I20260812 07:14:22.555764 22980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:22.557060 22980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:22.558251 22980 hybrid_clock.cc:648] HybridClock initialized: now 1786518862558203 us; error 36 us; skew 500 ppm
I20260812 07:14:22.558531 22980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:22 dist-test-slave-lgr5 krb5kdc[22953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518862, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:22.561439 22980 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:14:22.562748 22980 webserver.cc:460] Webserver started at http://127.21.181.126:40205/ using document root <none> and password file <none>
I20260812 07:14:22.563349 22980 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:22.563459 22980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:22.563750 22980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:22.566561 22980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "302dfb227c804490be0024402c87ab9b"
format_stamp: "Formatted at 2026-08-12 07:14:22 on dist-test-slave-lgr5"
server_key: "a9da3413efe483843f4e623819f32b20"
server_key_iv: "0742812167748fe14e375ee2d6266f4b"
server_key_version: "encryptionkey@0"
I20260812 07:14:22.567256 22980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "302dfb227c804490be0024402c87ab9b"
format_stamp: "Formatted at 2026-08-12 07:14:22 on dist-test-slave-lgr5"
server_key: "a9da3413efe483843f4e623819f32b20"
server_key_iv: "0742812167748fe14e375ee2d6266f4b"
server_key_version: "encryptionkey@0"
I20260812 07:14:22.572065 22980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 07:14:22.575238 23020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:22.576432 22980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:14:22.576593 22980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "302dfb227c804490be0024402c87ab9b"
format_stamp: "Formatted at 2026-08-12 07:14:22 on dist-test-slave-lgr5"
server_key: "a9da3413efe483843f4e623819f32b20"
server_key_iv: "0742812167748fe14e375ee2d6266f4b"
server_key_version: "encryptionkey@0"
I20260812 07:14:22.576762 22980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:22.595221 22980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:22.598858 22980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:22.599112 22980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:22.615808 22980 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:38513
I20260812 07:14:22.615829 23122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:38513 every 8 connection(s)
I20260812 07:14:22.617013 22980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:22.618615 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 22980
I20260812 07:14:22.618790 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:22.619246 22229 external_mini_cluster.cc:1428] Setting key 83f01e39c5cea9ae1564481233d9010a
I20260812 07:14:22.620687 23124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:14:22 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:22.633359 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b: Bootstrap starting.
I20260812 07:14:22.636466 23124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:22.637441 23124 log.cc:824] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:22.640424 23124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b: No bootstrap required, opened a new log
I20260812 07:14:22.641183 23131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:22.620905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.21.181.126:38513)
0812 07:14:22.621623 (+ 718us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:22.621638 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:22.621685 (+ 47us) server_negotiation.cc:407] Connection header received
0812 07:14:22.623634 (+ 1949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:22.623659 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:22.624125 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:22.624544 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:22.625743 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:22.627719 (+ 1976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:22.629092 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:22.630073 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:22.630271 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:22.634383 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:22.634390 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:22.634402 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:22.634440 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:22.637336 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:22.638169 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:22.638177 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:22.638185 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:22.638398 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:22.638771 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:22.638774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:22.638776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:22.639260 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:22.639472 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:22.639895 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 07:14:22.640251 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":290,"threads_started":1}
I20260812 07:14:22.647655 23124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "302dfb227c804490be0024402c87ab9b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38513 } }
I20260812 07:14:22.647956 23124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:22.648066 23124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 302dfb227c804490be0024402c87ab9b, State: Initialized, Role: FOLLOWER
I20260812 07:14:22.648620 23124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "302dfb227c804490be0024402c87ab9b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38513 } }
I20260812 07:14:22.648847 23124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:22.648996 23124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:22.649156 23124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:22.651077 23124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "302dfb227c804490be0024402c87ab9b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38513 } }
I20260812 07:14:22.651466 23124 leader_election.cc:302] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 302dfb227c804490be0024402c87ab9b; no voters:
I20260812 07:14:22.651943 23124 leader_election.cc:288] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:22.652060 23133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:22.652694 23133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [term 1 LEADER]: Becoming Leader. State: Replica: 302dfb227c804490be0024402c87ab9b, State: Running, Role: LEADER
I20260812 07:14:22.653177 23133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "302dfb227c804490be0024402c87ab9b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38513 } }
I20260812 07:14:22.653733 23124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:22.655791 23135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "302dfb227c804490be0024402c87ab9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "302dfb227c804490be0024402c87ab9b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38513 } } }
I20260812 07:14:22.655797 23136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 302dfb227c804490be0024402c87ab9b. Latest consensus state: current_term: 1 leader_uuid: "302dfb227c804490be0024402c87ab9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "302dfb227c804490be0024402c87ab9b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38513 } } }
I20260812 07:14:22.656345 23135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:22.656375 23136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:22.657140 23144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:22.661892 23144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:22.674006 23144 catalog_manager.cc:1347] Generated new cluster ID: 39ba712bd3d745ed939a227956f3dbf6
I20260812 07:14:22.674105 23144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:22.701367 23144 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:22.703081 23144 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:22.710387 23144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b: Generated new TSK 0
I20260812 07:14:22.711262 23144 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:14:22.780745 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38513
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:23.101014 23177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:23.101262 23177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:23.101387 23177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:23.108389 23177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:23.108484 23177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:23.108629 23177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:14:23.118094 23177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:23.119298 23177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:23.120604 23177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:23.132898 23190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:23.132946 23191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:23.132994 23201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:23.133757 23177 server_base.cc:1034] running on GCE node
I20260812 07:14:23.134298 23177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:23.135125 23177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:23.136305 23177 hybrid_clock.cc:648] HybridClock initialized: now 1786518863136250 us; error 54 us; skew 500 ppm
I20260812 07:14:23.136590 23177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:23 dist-test-slave-lgr5 krb5kdc[22953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:23.139595 23177 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:14:23.141108 23177 webserver.cc:460] Webserver started at http://127.21.181.65:33843/ using document root <none> and password file <none>
I20260812 07:14:23.141760 23177 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:23.141891 23177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:23.142179 23177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:23.144915 23177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "5a806be06e394dcdbef39e98000c7549"
format_stamp: "Formatted at 2026-08-12 07:14:23 on dist-test-slave-lgr5"
server_key: "51bf0876f742f1d900c1b58c7802cde2"
server_key_iv: "8da5877f510c6c6a53f20c77bec7549a"
server_key_version: "encryptionkey@0"
I20260812 07:14:23.145635 23177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "5a806be06e394dcdbef39e98000c7549"
format_stamp: "Formatted at 2026-08-12 07:14:23 on dist-test-slave-lgr5"
server_key: "51bf0876f742f1d900c1b58c7802cde2"
server_key_iv: "8da5877f510c6c6a53f20c77bec7549a"
server_key_version: "encryptionkey@0"
I20260812 07:14:23.150481 23177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:23.153724 23213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:23.155112 23177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:14:23.155254 23177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "5a806be06e394dcdbef39e98000c7549"
format_stamp: "Formatted at 2026-08-12 07:14:23 on dist-test-slave-lgr5"
server_key: "51bf0876f742f1d900c1b58c7802cde2"
server_key_iv: "8da5877f510c6c6a53f20c77bec7549a"
server_key_version: "encryptionkey@0"
I20260812 07:14:23.155423 23177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:23.176828 23177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:23.180302 23177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:23.180547 23177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:23.181429 23177 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:23.182634 23177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:23.182723 23177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:23.182842 23177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:23.182896 23177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:23.212790 23177 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:36245
I20260812 07:14:23.212811 23454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:36245 every 8 connection(s)
I20260812 07:14:23.214036 23177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:14:23.215557 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 23177
I20260812 07:14:23.215731 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:14:23.216288 22229 external_mini_cluster.cc:1428] Setting key 7b95225cdd68dbf32aeb9fa65228e7c8
Aug 12 07:14:23 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:23.234055 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:23.216686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:48153 (local address 127.21.181.126:38513)
0812 07:14:23.217133 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:23.217138 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:23.218209 (+ 1071us) server_negotiation.cc:407] Connection header received
0812 07:14:23.220006 (+ 1797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:23.220012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:23.220112 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:23.220216 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:23.221835 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.222745 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.224267 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.224790 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.224914 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:23.228222 (+ 3308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:23.228228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:23.228232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:23.228268 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:23.231098 (+ 2830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:23.231897 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:23.231901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:23.231904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:23.232027 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:23.232532 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:23.232535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:23.232537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:23.232787 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:23.232892 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:23.233651 (+ 759us) server_negotiation.cc:299] Negotiation successful
0812 07:14:23.233809 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260812 07:14:23.235325 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:23.217231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38513 (local address 127.21.181.65:48153)
0812 07:14:23.218039 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:23.218088 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:23.219753 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:23.220400 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:23.220408 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:23.220889 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:23.221621 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:23.221651 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.222901 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.222905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:23.224121 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:23.224137 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.224967 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.224972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:23.225170 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:23.225784 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:23.225806 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:23.227982 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:23.231217 (+ 3235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:23.231224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:23.231235 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:23.231764 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:23.232184 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:23.232189 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:23.232192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:23.232427 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:23.232922 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:23.232929 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:23.233293 (+ 364us) client_negotiation.cc:769] Sending connection context
0812 07:14:23.233576 (+ 283us) client_negotiation.cc:240] Negotiation successful
0812 07:14:23.234046 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":283,"threads_started":1}
I20260812 07:14:23.236819 23455 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38513
I20260812 07:14:23.237106 23455 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:23.237752 23455 heartbeater.cc:511] Master 127.21.181.126:38513 requested a full tablet report, sending...
I20260812 07:14:23.239705 23056 ts_manager.cc:194] Registered new tserver with Master: 5a806be06e394dcdbef39e98000c7549 (127.21.181.65:36245)
I20260812 07:14:23.241122 23056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:48153
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:14:23.279273 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38513
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:23.592928 23465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:23.593176 23465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:23.593307 23465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:23.600342 23465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:23.600433 23465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:23.600586 23465 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:14:23.609783 23465 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:23.610893 23465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:23.612102 23465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:23.624327 23478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:23.624631 23483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:23.624902 23477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:23.625663 23465 server_base.cc:1034] running on GCE node
I20260812 07:14:23.626180 23465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:23.627049 23465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:23.628239 23465 hybrid_clock.cc:648] HybridClock initialized: now 1786518863628190 us; error 44 us; skew 500 ppm
I20260812 07:14:23.628517 23465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:23 dist-test-slave-lgr5 krb5kdc[22953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:23.631482 23465 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:14:23.632866 23465 webserver.cc:460] Webserver started at http://127.21.181.66:37587/ using document root <none> and password file <none>
I20260812 07:14:23.633471 23465 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:23.633586 23465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:23.633882 23465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:23.636591 23465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "e14339370ddc4b4b805397ad07f63d58"
format_stamp: "Formatted at 2026-08-12 07:14:23 on dist-test-slave-lgr5"
server_key: "5de433a9ef259512fe286cbbe9ddce8b"
server_key_iv: "aef970a7b542d0d289ac18b67100b6a8"
server_key_version: "encryptionkey@0"
I20260812 07:14:23.637296 23465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "e14339370ddc4b4b805397ad07f63d58"
format_stamp: "Formatted at 2026-08-12 07:14:23 on dist-test-slave-lgr5"
server_key: "5de433a9ef259512fe286cbbe9ddce8b"
server_key_iv: "aef970a7b542d0d289ac18b67100b6a8"
server_key_version: "encryptionkey@0"
I20260812 07:14:23.642019 23465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:23.645220 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:23.646366 23465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:14:23.646509 23465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "e14339370ddc4b4b805397ad07f63d58"
format_stamp: "Formatted at 2026-08-12 07:14:23 on dist-test-slave-lgr5"
server_key: "5de433a9ef259512fe286cbbe9ddce8b"
server_key_iv: "aef970a7b542d0d289ac18b67100b6a8"
server_key_version: "encryptionkey@0"
I20260812 07:14:23.646682 23465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:23.669112 23465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:23.672488 23465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:23.672729 23465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:23.673623 23465 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:23.674821 23465 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:23.674906 23465 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:23.675019 23465 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:23.675086 23465 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:23.704274 23465 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:32835
I20260812 07:14:23.704314 23716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:32835 every 8 connection(s)
I20260812 07:14:23.705582 23465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:14:23.712338 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 23465
I20260812 07:14:23.712530 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:14:23.713076 22229 external_mini_cluster.cc:1428] Setting key 77ce1983c50fbf38d4024691c3f7e4a1
Aug 12 07:14:23 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518863, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:23.725165 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:23.707954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:46711 (local address 127.21.181.126:38513)
0812 07:14:23.708149 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:23.708153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:23.709225 (+ 1072us) server_negotiation.cc:407] Connection header received
0812 07:14:23.711046 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:23.711052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:23.711133 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:23.711224 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:23.712985 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.713816 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.715669 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.716104 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.716209 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:23.719990 (+ 3781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:23.719995 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:23.719998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:23.720033 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:23.722354 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:23.723113 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:23.723116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:23.723118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:23.723238 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:23.723658 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:23.723661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:23.723663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:23.723901 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:23.724003 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:23.724760 (+ 757us) server_negotiation.cc:299] Negotiation successful
0812 07:14:23.724923 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:14:23.726349 23722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:23.708374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38513 (local address 127.21.181.66:46711)
0812 07:14:23.709064 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:23.709100 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:23.710774 (+ 1674us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:23.711457 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:23.711465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:23.711959 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:23.712772 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:23.712800 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.713976 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.713982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:23.715532 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:23.715547 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:23.716321 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:23.716328 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:23.716608 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:23.717273 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:23.717302 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:23.719759 (+ 2457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:23.722488 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:23.722504 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:23.722517 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:23.723019 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:23.723327 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:23.723331 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:23.723335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:23.723570 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:23.724013 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:23.724020 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:23.724405 (+ 385us) client_negotiation.cc:769] Sending connection context
0812 07:14:23.724698 (+ 293us) client_negotiation.cc:240] Negotiation successful
0812 07:14:23.725142 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":252,"threads_started":1}
I20260812 07:14:23.727690 23718 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38513
I20260812 07:14:23.727994 23718 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:23.728833 23718 heartbeater.cc:511] Master 127.21.181.126:38513 requested a full tablet report, sending...
I20260812 07:14:23.730206 23056 ts_manager.cc:194] Registered new tserver with Master: e14339370ddc4b4b805397ad07f63d58 (127.21.181.66:32835)
I20260812 07:14:23.730957 23056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:46711
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:14:23.772969 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38513
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:39039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:24.088457 23725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:24.088717 23725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:24.088851 23725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:24.095784 23725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:24.095880 23725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:24.096035 23725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:14:24.105325 23725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:39039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:38513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:24.106444 23725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:24.107676 23725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:24.119386 23734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:24.119426 23735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:24.119465 23743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:24.120682 23725 server_base.cc:1034] running on GCE node
I20260812 07:14:24.121205 23725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:24.122046 23725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:24.123229 23725 hybrid_clock.cc:648] HybridClock initialized: now 1786518864123189 us; error 33 us; skew 500 ppm
I20260812 07:14:24.123521 23725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:24 dist-test-slave-lgr5 krb5kdc[22953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:24.126601 23725 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:14:24.127913 23725 webserver.cc:460] Webserver started at http://127.21.181.67:39205/ using document root <none> and password file <none>
I20260812 07:14:24.128556 23725 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:24.128651 23725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:24.128906 23725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:24.131567 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "70354ffe53cb4e97aa7faa54f241026d"
format_stamp: "Formatted at 2026-08-12 07:14:24 on dist-test-slave-lgr5"
server_key: "881f08565d42a07b6bc2a5ca6526f2a7"
server_key_iv: "e9b719fa4d3ff46543c8ffd5c46c046f"
server_key_version: "encryptionkey@0"
I20260812 07:14:24.132346 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "70354ffe53cb4e97aa7faa54f241026d"
format_stamp: "Formatted at 2026-08-12 07:14:24 on dist-test-slave-lgr5"
server_key: "881f08565d42a07b6bc2a5ca6526f2a7"
server_key_iv: "e9b719fa4d3ff46543c8ffd5c46c046f"
server_key_version: "encryptionkey@0"
I20260812 07:14:24.136994 23725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:14:24.140174 23763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:24.141379 23725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:14:24.141508 23725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "70354ffe53cb4e97aa7faa54f241026d"
format_stamp: "Formatted at 2026-08-12 07:14:24 on dist-test-slave-lgr5"
server_key: "881f08565d42a07b6bc2a5ca6526f2a7"
server_key_iv: "e9b719fa4d3ff46543c8ffd5c46c046f"
server_key_version: "encryptionkey@0"
I20260812 07:14:24.141656 23725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:24.157208 23725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:24.160983 23725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:24.161206 23725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:24.162037 23725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:24.163287 23725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:24.163367 23725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:24.163473 23725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:24.163516 23725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:24.193982 23725 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:38853
I20260812 07:14:24.194028 23986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:38853 every 8 connection(s)
I20260812 07:14:24.195197 23725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:14:24.204038 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 23725
I20260812 07:14:24.204245 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:14:24.204843 22229 external_mini_cluster.cc:1428] Setting key a235227c77688a5141e88fe04f0cd88d
Aug 12 07:14:24 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518864, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:24.220973 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.200776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:50865 (local address 127.21.181.126:38513)
0812 07:14:24.201219 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.201225 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.203529 (+ 2304us) server_negotiation.cc:407] Connection header received
0812 07:14:24.206204 (+ 2675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.206211 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.206299 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.206405 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:24.208249 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.209338 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.210828 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.211467 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.211597 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.214767 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:24.214772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:24.214775 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:24.214809 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:24.217364 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.218294 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.218299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.218302 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.218469 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.218982 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.218987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.218990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.219365 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:24.219515 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.220413 (+ 898us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.220652 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301}
I20260812 07:14:24.221851 23990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.198408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:38513 (local address 127.21.181.67:50865)
0812 07:14:24.203357 (+ 4949us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:24.203411 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:24.205839 (+ 2428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:24.206835 (+ 996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:24.206843 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:24.207385 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:24.208050 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.208071 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.209466 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.209470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.210673 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.210687 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.211656 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.211660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.211845 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.212512 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:24.212534 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:24.214572 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:24.217498 (+ 2926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:24.217507 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:24.217521 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:24.217908 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:24.218615 (+ 707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:24.218618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:24.218621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:24.218851 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:24.219598 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:24.219603 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:24.219956 (+ 353us) client_negotiation.cc:769] Sending connection context
0812 07:14:24.220277 (+ 321us) client_negotiation.cc:240] Negotiation successful
0812 07:14:24.220727 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4667,"thread_start_us":4421,"threads_started":1}
I20260812 07:14:24.223199 23988 heartbeater.cc:347] Connected to a master server at 127.21.181.126:38513
I20260812 07:14:24.223465 23988 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:24.224189 23988 heartbeater.cc:511] Master 127.21.181.126:38513 requested a full tablet report, sending...
I20260812 07:14:24.225662 23056 ts_manager.cc:194] Registered new tserver with Master: 70354ffe53cb4e97aa7faa54f241026d (127.21.181.67:38853)
I20260812 07:14:24.226747 23056 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:50865
I20260812 07:14:24.232124 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:14:24.243830 23455 heartbeater.cc:503] Master 127.21.181.126:38513 was elected leader, sending a full tablet report...
I20260812 07:14:24.249554 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.236267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.21.181.126:38513)
0812 07:14:24.236419 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.236423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.236922 (+ 499us) server_negotiation.cc:407] Connection header received
0812 07:14:24.237087 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.237093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.237156 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.237235 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:24.238633 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.239330 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.240678 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.241070 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.241178 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.243689 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:24.243695 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:24.243701 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:24.243734 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:24.246637 (+ 2903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.247388 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.247393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.247397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.247521 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.247961 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.247967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.247970 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.248334 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:24.248582 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.249025 (+ 443us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.249258 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:14:24.253175 23056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:14:24.260341 23056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:14:24.301168 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.273336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54024 (local address 127.21.181.65:36245)
0812 07:14:24.282193 (+ 8857us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.282201 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.292199 (+ 9998us) server_negotiation.cc:407] Connection header received
0812 07:14:24.292350 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.292358 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.292639 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.292858 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:24.293869 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.295406 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.298171 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.299603 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.299743 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.299978 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.300657 (+ 679us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.300817 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8679,"thread_start_us":9128,"threads_started":1}
I20260812 07:14:24.301203 24007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.272422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36245 (local address 127.0.0.1:54024)
0812 07:14:24.281407 (+ 8985us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:24.281442 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:24.281617 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:24.293086 (+ 11469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:24.293095 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:24.293136 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:24.293484 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.293498 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.295597 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.295602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.297977 (+ 2375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.297993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.300183 (+ 2190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.300189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.300384 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.300437 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 07:14:24.300565 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 07:14:24.300759 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8751,"mutex_wait_us":2,"thread_start_us":16539,"threads_started":1}
I20260812 07:14:24.303507 24011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.272819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:32835 (local address 127.0.0.1:39474)
0812 07:14:24.288481 (+ 15662us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:24.288499 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:24.288625 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:24.294062 (+ 5437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:24.294067 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:24.294110 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:24.294645 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.294659 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.297013 (+ 2354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.297019 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.299321 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.299337 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.302627 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.302634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.302807 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.302860 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 07:14:24.302956 (+ 96us) client_negotiation.cc:240] Negotiation successful
0812 07:14:24.303086 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15541,"mutex_wait_us":1,"spinlock_wait_cycles":2816,"thread_start_us":8227,"threads_started":1}
I20260812 07:14:24.303872 24006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.282328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38853 (local address 127.0.0.1:44380)
0812 07:14:24.289211 (+ 6883us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:24.289229 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:24.289341 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:24.294070 (+ 4729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:24.294075 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:24.294124 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:24.294625 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.294640 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.296786 (+ 2146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.296791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.299842 (+ 3051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.299856 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.302930 (+ 3074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.302937 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.303108 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.303139 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:14:24.303213 (+ 74us) client_negotiation.cc:240] Negotiation successful
0812 07:14:24.303329 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6756,"mutex_wait_us":922,"spinlock_wait_cycles":1920,"thread_start_us":6367,"threads_started":1}
I20260812 07:14:24.304648 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.272859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39474 (local address 127.21.181.66:32835)
0812 07:14:24.293271 (+ 20412us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.293280 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.293312 (+ 32us) server_negotiation.cc:407] Connection header received
0812 07:14:24.293413 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.293420 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.293651 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.294861 (+ 1210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:24.295149 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.296828 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.301392 (+ 4564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.302408 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.302551 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.302741 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.304074 (+ 1333us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.304286 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20200,"thread_start_us":20939,"threads_started":1}
I20260812 07:14:24.304695 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.282580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44380 (local address 127.21.181.67:38853)
0812 07:14:24.293489 (+ 10909us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.293497 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.293531 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:14:24.293640 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.293648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.293857 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.294851 (+ 994us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:24.295109 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.296628 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.300014 (+ 3386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.301140 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.303779 (+ 2639us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.304135 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.304212 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.304363 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10735,"thread_start_us":11026,"threads_started":1}
I20260812 07:14:24.305615 23304 tablet_service.cc:1467] Processing CreateTablet for tablet 102553db158e43238bf61c0b843948e8 (DEFAULT_TABLE table=test-table [id=6f74db20a91e46079b9c6e919072ef02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:24.307317 23304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102553db158e43238bf61c0b843948e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:24.308024 23588 tablet_service.cc:1467] Processing CreateTablet for tablet 102553db158e43238bf61c0b843948e8 (DEFAULT_TABLE table=test-table [id=6f74db20a91e46079b9c6e919072ef02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:24.308354 23860 tablet_service.cc:1467] Processing CreateTablet for tablet 102553db158e43238bf61c0b843948e8 (DEFAULT_TABLE table=test-table [id=6f74db20a91e46079b9c6e919072ef02]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:24.309466 23588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102553db158e43238bf61c0b843948e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:24.309679 23860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 102553db158e43238bf61c0b843948e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:24.326872 24014 tablet_bootstrap.cc:492] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Bootstrap starting.
I20260812 07:14:24.329998 24020 tablet_bootstrap.cc:492] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Bootstrap starting.
I20260812 07:14:24.330052 24014 tablet_bootstrap.cc:654] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:24.331656 24014 log.cc:824] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:24.332813 24018 tablet_bootstrap.cc:492] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Bootstrap starting.
I20260812 07:14:24.333455 24020 tablet_bootstrap.cc:654] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:24.334887 24020 log.cc:824] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:24.335914 24014 tablet_bootstrap.cc:492] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: No bootstrap required, opened a new log
I20260812 07:14:24.336225 24018 tablet_bootstrap.cc:654] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:24.336216 24014 ts_tablet_manager.cc:1397] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260812 07:14:24.337427 24018 log.cc:824] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:24.346493 24018 tablet_bootstrap.cc:492] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: No bootstrap required, opened a new log
I20260812 07:14:24.346752 24018 ts_tablet_manager.cc:1397] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.010s
I20260812 07:14:24.347260 24020 tablet_bootstrap.cc:492] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: No bootstrap required, opened a new log
I20260812 07:14:24.347561 24020 ts_tablet_manager.cc:1397] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.008s
I20260812 07:14:24.347601 24014 raft_consensus.cc:348] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.348033 24014 raft_consensus.cc:374] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:24.348174 24014 raft_consensus.cc:729] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a806be06e394dcdbef39e98000c7549, State: Initialized, Role: FOLLOWER
I20260812 07:14:24.348760 24014 consensus_queue.cc:260] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.358285 24018 raft_consensus.cc:348] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.358740 24018 raft_consensus.cc:374] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:24.358855 24018 raft_consensus.cc:729] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e14339370ddc4b4b805397ad07f63d58, State: Initialized, Role: FOLLOWER
I20260812 07:14:24.358465 24020 raft_consensus.cc:348] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.359130 24020 raft_consensus.cc:374] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:24.359256 24020 raft_consensus.cc:729] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70354ffe53cb4e97aa7faa54f241026d, State: Initialized, Role: FOLLOWER
I20260812 07:14:24.360147 24020 consensus_queue.cc:260] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.360574 24014 ts_tablet_manager.cc:1428] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Time spent starting tablet: real 0.024s user 0.013s sys 0.010s
I20260812 07:14:24.361351 24018 consensus_queue.cc:260] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.367539 23988 heartbeater.cc:503] Master 127.21.181.126:38513 was elected leader, sending a full tablet report...
I20260812 07:14:24.368496 24020 ts_tablet_manager.cc:1428] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Time spent starting tablet: real 0.021s user 0.015s sys 0.003s
I20260812 07:14:24.369016 23718 heartbeater.cc:503] Master 127.21.181.126:38513 was elected leader, sending a full tablet report...
I20260812 07:14:24.369879 24018 ts_tablet_manager.cc:1428] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Time spent starting tablet: real 0.023s user 0.015s sys 0.006s
W20260812 07:14:24.446990 23989 tablet.cc:2369] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:14:24.457383 23721 tablet.cc:2369] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:14:24.466122 23458 tablet.cc:2369] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:14:24.675951 24025 raft_consensus.cc:482] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:14:24.676278 24025 raft_consensus.cc:504] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.677953 24025 leader_election.cc:288] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e14339370ddc4b4b805397ad07f63d58 (127.21.181.66:32835), 70354ffe53cb4e97aa7faa54f241026d (127.21.181.67:38853)
I20260812 07:14:24.684901 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.678326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:53541 (local address 127.21.181.66:32835)
0812 07:14:24.678506 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.678510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.678964 (+ 454us) server_negotiation.cc:407] Connection header received
0812 07:14:24.679194 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.679199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.679278 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.679377 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:24.680111 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.681098 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.683074 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.683886 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.684008 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.684060 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.684519 (+ 459us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.684650 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:14:24.684991 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.679218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:50875 (local address 127.21.181.67:38853)
0812 07:14:24.679398 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.679404 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.679735 (+ 331us) server_negotiation.cc:407] Connection header received
0812 07:14:24.679853 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.679858 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.679933 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.680040 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:24.680672 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.681510 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.683229 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.684020 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.684203 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.684275 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.684646 (+ 371us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.684779 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:14:24.685005 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.678217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:32835 (local address 127.21.181.65:53541)
0812 07:14:24.678861 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:24.678880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:24.679040 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:24.679522 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:24.679527 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:24.679553 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:24.679892 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.679905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.681221 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.681227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.682908 (+ 1681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.682921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.684131 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.684138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.684359 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.684384 (+ 25us) client_negotiation.cc:769] Sending connection context
0812 07:14:24.684460 (+ 76us) client_negotiation.cc:240] Negotiation successful
0812 07:14:24.684551 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":112,"thread_start_us":336,"threads_started":1}
I20260812 07:14:24.685114 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.678999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:38853 (local address 127.21.181.65:50875)
0812 07:14:24.679611 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:24.679625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:24.679736 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:24.680067 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:24.680072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:24.680115 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:24.680490 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.680514 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.681646 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.681650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.683071 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:24.683086 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.684279 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.684286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:24.684468 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.684510 (+ 42us) client_negotiation.cc:769] Sending connection context
0812 07:14:24.684600 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 07:14:24.684716 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":3,"thread_start_us":289,"threads_started":1}
I20260812 07:14:24.686046 23893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102553db158e43238bf61c0b843948e8" candidate_uuid: "5a806be06e394dcdbef39e98000c7549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70354ffe53cb4e97aa7faa54f241026d" is_pre_election: true
I20260812 07:14:24.686277 23617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102553db158e43238bf61c0b843948e8" candidate_uuid: "5a806be06e394dcdbef39e98000c7549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e14339370ddc4b4b805397ad07f63d58" is_pre_election: true
I20260812 07:14:24.686612 23893 raft_consensus.cc:2454] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a806be06e394dcdbef39e98000c7549 in term 0.
I20260812 07:14:24.686774 23617 raft_consensus.cc:2454] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a806be06e394dcdbef39e98000c7549 in term 0.
I20260812 07:14:24.687275 23217 leader_election.cc:302] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a806be06e394dcdbef39e98000c7549, 70354ffe53cb4e97aa7faa54f241026d; no voters:
I20260812 07:14:24.687628 24025 raft_consensus.cc:2781] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:14:24.687747 24025 raft_consensus.cc:482] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:14:24.687849 24025 raft_consensus.cc:3037] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:24.690047 24025 raft_consensus.cc:504] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.690623 24025 leader_election.cc:288] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [CANDIDATE]: Term 1 election: Requested vote from peers e14339370ddc4b4b805397ad07f63d58 (127.21.181.66:32835), 70354ffe53cb4e97aa7faa54f241026d (127.21.181.67:38853)
I20260812 07:14:24.691026 23617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102553db158e43238bf61c0b843948e8" candidate_uuid: "5a806be06e394dcdbef39e98000c7549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e14339370ddc4b4b805397ad07f63d58"
I20260812 07:14:24.691092 23893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "102553db158e43238bf61c0b843948e8" candidate_uuid: "5a806be06e394dcdbef39e98000c7549" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70354ffe53cb4e97aa7faa54f241026d"
I20260812 07:14:24.691236 23617 raft_consensus.cc:3037] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:24.691263 23893 raft_consensus.cc:3037] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:24.693385 23893 raft_consensus.cc:2454] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a806be06e394dcdbef39e98000c7549 in term 1.
I20260812 07:14:24.693528 23617 raft_consensus.cc:2454] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a806be06e394dcdbef39e98000c7549 in term 1.
I20260812 07:14:24.693763 23217 leader_election.cc:302] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a806be06e394dcdbef39e98000c7549, 70354ffe53cb4e97aa7faa54f241026d; no voters:
I20260812 07:14:24.694026 24025 raft_consensus.cc:2781] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:24.694430 24025 raft_consensus.cc:686] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 1 LEADER]: Becoming Leader. State: Replica: 5a806be06e394dcdbef39e98000c7549, State: Running, Role: LEADER
I20260812 07:14:24.694878 24025 consensus_queue.cc:237] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } }
I20260812 07:14:24.699455 23056 catalog_manager.cc:5495] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a806be06e394dcdbef39e98000c7549 (127.21.181.65). New cstate: current_term: 1 leader_uuid: "5a806be06e394dcdbef39e98000c7549" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a806be06e394dcdbef39e98000c7549" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 } health_report { overall_health: UNKNOWN } } }
I20260812 07:14:24.737020 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.731705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54036 (local address 127.21.181.65:36245)
0812 07:14:24.731884 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.731890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.732008 (+ 118us) server_negotiation.cc:407] Connection header received
0812 07:14:24.732224 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.732228 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.732298 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.732417 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:14:24.732832 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.733542 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.734891 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.735295 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.735402 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.735987 (+ 585us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:14:24.736468 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:14:24.736631 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.736765 (+ 134us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.736811 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260812 07:14:24.743944 23893 raft_consensus.cc:1261] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 1 FOLLOWER]: Refusing update from remote peer 5a806be06e394dcdbef39e98000c7549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:24.743993 23617 raft_consensus.cc:1261] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 1 FOLLOWER]: Refusing update from remote peer 5a806be06e394dcdbef39e98000c7549: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:24.744813 24034 consensus_queue.cc:1035] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70354ffe53cb4e97aa7faa54f241026d" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 38853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:24.745162 24025 consensus_queue.cc:1035] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e14339370ddc4b4b805397ad07f63d58" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 32835 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:24.756264 24039 mvcc.cc:205] Tried to move back new op lower bound from 7317581269983272960 to 7317581269793939456. Current Snapshot: MvccSnapshot[applied={T|T < 7317581269983272960}]
I20260812 07:14:24.766361 23055 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:24.766600 23055 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56904:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:24.769692 23055 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 302dfb227c804490be0024402c87ab9b: Sending DeleteTablet for 3 replicas of tablet 102553db158e43238bf61c0b843948e8
I20260812 07:14:24.770870 23588 tablet_service.cc:1514] Processing DeleteTablet for tablet 102553db158e43238bf61c0b843948e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:24 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:39474
I20260812 07:14:24.770900 23304 tablet_service.cc:1514] Processing DeleteTablet for tablet 102553db158e43238bf61c0b843948e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:24 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:54024
I20260812 07:14:24.771011 23860 tablet_service.cc:1514] Processing DeleteTablet for tablet 102553db158e43238bf61c0b843948e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:24 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:44380
I20260812 07:14:24.777439 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 23177
I20260812 07:14:24.782660 24052 tablet_replica.cc:330] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: stopping tablet replica
I20260812 07:14:24.783057 24052 raft_consensus.cc:2229] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:24.783542 24052 raft_consensus.cc:2258] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:24.785761 24055 tablet_replica.cc:330] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: stopping tablet replica
I20260812 07:14:24.786131 24055 raft_consensus.cc:2229] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:24.786584 24055 raft_consensus.cc:2258] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:24.786736 24054 tablet_replica.cc:330] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: stopping tablet replica
I20260812 07:14:24.787117 24054 raft_consensus.cc:2229] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:14:24.787681 24054 raft_consensus.cc:2258] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:24.788712 24055 ts_tablet_manager.cc:1905] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:24.788777 24052 ts_tablet_manager.cc:1905] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:24.789451 24054 ts_tablet_manager.cc:1905] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:24.795245 24055 ts_tablet_manager.cc:1918] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:24.795410 24055 log.cc:1196] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/102553db158e43238bf61c0b843948e8
I20260812 07:14:24.795455 24052 ts_tablet_manager.cc:1918] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:24.795622 24052 log.cc:1196] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/102553db158e43238bf61c0b843948e8
I20260812 07:14:24.795917 24055 ts_tablet_manager.cc:1939] T 102553db158e43238bf61c0b843948e8 P 70354ffe53cb4e97aa7faa54f241026d: Deleting consensus metadata
I20260812 07:14:24.796111 24052 ts_tablet_manager.cc:1939] T 102553db158e43238bf61c0b843948e8 P e14339370ddc4b4b805397ad07f63d58: Deleting consensus metadata
I20260812 07:14:24.796248 24054 ts_tablet_manager.cc:1918] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:24.796399 24054 log.cc:1196] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/102553db158e43238bf61c0b843948e8
I20260812 07:14:24.796866 24054 ts_tablet_manager.cc:1939] T 102553db158e43238bf61c0b843948e8 P 5a806be06e394dcdbef39e98000c7549: Deleting consensus metadata
I20260812 07:14:24.797686 23034 catalog_manager.cc:4841] TS e14339370ddc4b4b805397ad07f63d58 (127.21.181.66:32835): tablet 102553db158e43238bf61c0b843948e8 (table test-table [id=6f74db20a91e46079b9c6e919072ef02]) successfully deleted
I20260812 07:14:24.797997 23034 catalog_manager.cc:4841] TS 70354ffe53cb4e97aa7faa54f241026d (127.21.181.67:38853): tablet 102553db158e43238bf61c0b843948e8 (table test-table [id=6f74db20a91e46079b9c6e919072ef02]) successfully deleted
I20260812 07:14:24.798296 23029 catalog_manager.cc:4841] TS 5a806be06e394dcdbef39e98000c7549 (127.21.181.65:36245): tablet 102553db158e43238bf61c0b843948e8 (table test-table [id=6f74db20a91e46079b9c6e919072ef02]) successfully deleted
Aug 12 07:14:24 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:14:24.809765 24009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.779203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54052 (local address 127.21.181.65:36245)
0812 07:14:24.782223 (+ 3020us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.782228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.787704 (+ 5476us) server_negotiation.cc:407] Connection header received
0812 07:14:24.788991 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.788997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.789075 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.789148 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:24.795478 (+ 6330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.796677 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.798857 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.799352 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.799452 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.804256 (+ 4804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:24.804267 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:24.804281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:24.804311 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:24.807050 (+ 2739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.807713 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.807719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.807721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.807869 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.808294 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.808297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.808299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.808653 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:24.808949 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.809246 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.809460 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2873}
I20260812 07:14:24.810369 23442 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:24.909618 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 23177
I20260812 07:14:24.932202 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 23465
Aug 12 07:14:24 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:14:24.946848 24008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:24.932985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.21.181.66:32835)
0812 07:14:24.933168 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:24.933175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:24.933196 (+ 21us) server_negotiation.cc:407] Connection header received
0812 07:14:24.933365 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:24.933369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:24.933433 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:24.933518 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:24.934757 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.935499 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.936889 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:24.937284 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:24.937400 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:24.941373 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:24.941386 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:24.941400 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:24.941436 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:24.944044 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.944781 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.944790 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.944792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.944936 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:24.945315 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:24.945321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:24.945325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:24.945704 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:24.945984 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:24.946300 (+ 316us) server_negotiation.cc:299] Negotiation successful
0812 07:14:24.946521 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:14:24.947451 23708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:25.035167 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 23465
I20260812 07:14:25.062525 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 23725
Aug 12 07:14:25 dist-test-slave-lgr5 krb5kdc[22953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:14:25.077113 24013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:25.063249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.21.181.67:38853)
0812 07:14:25.063435 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:25.063441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:25.063461 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:14:25.063661 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:25.063668 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:25.063747 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:25.063831 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:25.065279 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:25.066026 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:25.067282 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:25.067687 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:25.067800 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:25.071815 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:25.071830 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:25.071847 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:25.071882 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:25.074355 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:25.075016 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:25.075023 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:25.075026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:25.075177 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:25.075550 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:25.075554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:25.075556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:25.075957 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:25.076268 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:25.076578 (+ 310us) server_negotiation.cc:299] Negotiation successful
0812 07:14:25.076787 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260812 07:14:25.077739 23978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:25.164211 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 23725
I20260812 07:14:25.188671 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 22980
I20260812 07:14:25.189724 23109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:25.296327 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 22980
2026-08-12T07:14:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4596 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-12T07:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:25Z Disabled control of system clock
I20260812 07:14:25.385876 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:40809
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:40809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:14:25.701551 24068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:25.701800 24068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:25.701891 24068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:25.708725 24068 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:25.708812 24068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:25.708861 24068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:25.708915 24068 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:25.708957 24068 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:25.717577 24068 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:40809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:40809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:25.718605 24068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:25.720237 24068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:25.730965 24079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:25.730929 24083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:25.731192 24078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:25.731698 24068 server_base.cc:1034] running on GCE node
I20260812 07:14:25.732422 24068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:25.733655 24068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:25.734841 24068 hybrid_clock.cc:648] HybridClock initialized: now 1786518865734785 us; error 47 us; skew 500 ppm
I20260812 07:14:25.735155 24068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:25.740901 24068 webserver.cc:460] Webserver started at http://127.21.181.126:40555/ using document root <none> and password file <none>
I20260812 07:14:25.741537 24068 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:25.741631 24068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:25.741947 24068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:25.744727 24068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "56a50246d85a4ad7bcd57d835dee5131"
format_stamp: "Formatted at 2026-08-12 07:14:25 on dist-test-slave-lgr5"
server_key: "2060f4b855eec5703e306e5b6c388801"
server_key_iv: "fcd3d40fcfdbdb3a997e847ac446fdf5"
server_key_version: "encryptionkey@0"
I20260812 07:14:25.745491 24068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "56a50246d85a4ad7bcd57d835dee5131"
format_stamp: "Formatted at 2026-08-12 07:14:25 on dist-test-slave-lgr5"
server_key: "2060f4b855eec5703e306e5b6c388801"
server_key_iv: "fcd3d40fcfdbdb3a997e847ac446fdf5"
server_key_version: "encryptionkey@0"
I20260812 07:14:25.750241 24068 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:14:25.753434 24096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:25.754637 24068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:14:25.754792 24068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "56a50246d85a4ad7bcd57d835dee5131"
format_stamp: "Formatted at 2026-08-12 07:14:25 on dist-test-slave-lgr5"
server_key: "2060f4b855eec5703e306e5b6c388801"
server_key_iv: "fcd3d40fcfdbdb3a997e847ac446fdf5"
server_key_version: "encryptionkey@0"
I20260812 07:14:25.754985 24068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:25.771245 24068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:25.771947 24068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:25.772217 24068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:25.788722 24068 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:40809
I20260812 07:14:25.788761 24183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:40809 every 8 connection(s)
I20260812 07:14:25.789885 24068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:25.793304 24184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:25.797660 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24068
I20260812 07:14:25.797832 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:25.798369 22229 external_mini_cluster.cc:1428] Setting key 0a4ade927fc4ef5a141a44714612a22b
I20260812 07:14:25.803953 24184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131: Bootstrap starting.
I20260812 07:14:25.806823 24184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:25.807813 24184 log.cc:824] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:25.810611 24184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131: No bootstrap required, opened a new log
I20260812 07:14:25.810926 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:25.800213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49536 (local address 127.21.181.126:40809)
0812 07:14:25.801135 (+ 922us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:25.801145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:25.801187 (+ 42us) server_negotiation.cc:407] Connection header received
0812 07:14:25.802875 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:25.802898 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:25.803277 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:25.803594 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:25.804174 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:25.805423 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:25.806722 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:25.807426 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:25.807595 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:25.808778 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:25.808788 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:25.808809 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:25.809371 (+ 562us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:25.809448 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:25.809469 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:25.809551 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:25.809766 (+ 215us) server_negotiation.cc:299] Negotiation successful
0812 07:14:25.810050 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":328,"threads_started":1}
I20260812 07:14:25.817610 24184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a50246d85a4ad7bcd57d835dee5131" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40809 } }
I20260812 07:14:25.817919 24184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:25.818013 24184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a50246d85a4ad7bcd57d835dee5131, State: Initialized, Role: FOLLOWER
I20260812 07:14:25.818490 24184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a50246d85a4ad7bcd57d835dee5131" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40809 } }
I20260812 07:14:25.818701 24184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:25.818820 24184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:25.818971 24184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:25.821015 24184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a50246d85a4ad7bcd57d835dee5131" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40809 } }
I20260812 07:14:25.821462 24184 leader_election.cc:302] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a50246d85a4ad7bcd57d835dee5131; no voters:
I20260812 07:14:25.821955 24184 leader_election.cc:288] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:25.822079 24193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:25.822698 24193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [term 1 LEADER]: Becoming Leader. State: Replica: 56a50246d85a4ad7bcd57d835dee5131, State: Running, Role: LEADER
I20260812 07:14:25.823136 24193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a50246d85a4ad7bcd57d835dee5131" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40809 } }
I20260812 07:14:25.823763 24184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:25.825536 24198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a50246d85a4ad7bcd57d835dee5131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a50246d85a4ad7bcd57d835dee5131" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40809 } } }
I20260812 07:14:25.825644 24199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a50246d85a4ad7bcd57d835dee5131. Latest consensus state: current_term: 1 leader_uuid: "56a50246d85a4ad7bcd57d835dee5131" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a50246d85a4ad7bcd57d835dee5131" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 40809 } } }
I20260812 07:14:25.825971 24198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:25.826016 24199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:25.826921 24209 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:25.832269 24209 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:25.840178 24209 catalog_manager.cc:1347] Generated new cluster ID: 9a41acaa716048fa92b9bdd80fc73631
I20260812 07:14:25.840273 24209 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:25.852564 24209 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:25.854204 24209 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:25.874938 24209 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131: Generated new TSK 0
I20260812 07:14:25.875898 24209 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:25.882247 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:40809
--builtin_ntp_servers=127.21.181.84:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:14:26.219961 24225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:26.220256 24225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:26.220399 24225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:26.227684 24225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:26.227795 24225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:26.227968 24225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:14:26.236958 24225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:40809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:26.238173 24225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:26.239938 24225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:26.252651 24235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:26.252688 24234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:26.253037 24237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:26.253724 24225 server_base.cc:1034] running on GCE node
I20260812 07:14:26.254302 24225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:26.255213 24225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:26.256433 24225 hybrid_clock.cc:648] HybridClock initialized: now 1786518866256374 us; error 35 us; skew 500 ppm
I20260812 07:14:26.256750 24225 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:26.259057 24225 webserver.cc:460] Webserver started at http://127.21.181.65:40361/ using document root <none> and password file <none>
I20260812 07:14:26.259719 24225 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:26.259826 24225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:26.260133 24225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:26.262977 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "5fa4d71171a94a51913ef98893604b2e"
format_stamp: "Formatted at 2026-08-12 07:14:26 on dist-test-slave-lgr5"
server_key: "3b6b55fa4bd224f264606f7d2555e812"
server_key_iv: "a7166981214728383291193a083d9a0c"
server_key_version: "encryptionkey@0"
I20260812 07:14:26.263765 24225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "5fa4d71171a94a51913ef98893604b2e"
format_stamp: "Formatted at 2026-08-12 07:14:26 on dist-test-slave-lgr5"
server_key: "3b6b55fa4bd224f264606f7d2555e812"
server_key_iv: "a7166981214728383291193a083d9a0c"
server_key_version: "encryptionkey@0"
I20260812 07:14:26.269032 24225 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260812 07:14:26.273442 24246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:26.274724 24225 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260812 07:14:26.274892 24225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "5fa4d71171a94a51913ef98893604b2e"
format_stamp: "Formatted at 2026-08-12 07:14:26 on dist-test-slave-lgr5"
server_key: "3b6b55fa4bd224f264606f7d2555e812"
server_key_iv: "a7166981214728383291193a083d9a0c"
server_key_version: "encryptionkey@0"
I20260812 07:14:26.275643 24225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:26.291869 24225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:26.292693 24225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:26.293002 24225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:26.293956 24225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:26.295444 24225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:26.295559 24225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:26.295672 24225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:26.295751 24225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:26.326229 24225 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:36007
I20260812 07:14:26.326288 24445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:36007 every 8 connection(s)
I20260812 07:14:26.327595 24225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:14:26.331542 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24225
I20260812 07:14:26.331692 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:14:26.332149 22229 external_mini_cluster.cc:1428] Setting key 11417fd061f80ed84e4a45570f7fc238
I20260812 07:14:26.335089 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:40809
--builtin_ntp_servers=127.21.181.84:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:14:26.339643 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:26.329971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:58947 (local address 127.21.181.126:40809)
0812 07:14:26.330379 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:26.330384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:26.331424 (+ 1040us) server_negotiation.cc:407] Connection header received
0812 07:14:26.333329 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:26.333333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:26.333397 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:26.333485 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:26.334571 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.335550 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.337097 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.337612 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.337727 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:26.338869 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:26.338875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:26.338878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:26.338897 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:26.338949 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:26.338963 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:26.339052 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:26.339205 (+ 153us) server_negotiation.cc:299] Negotiation successful
0812 07:14:26.339324 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":297,"thread_start_us":201,"threads_started":1}
I20260812 07:14:26.340436 24453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:26.330443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:40809 (local address 127.21.181.65:58947)
0812 07:14:26.331268 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:26.331303 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:26.333053 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:26.333744 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:26.333755 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:26.333875 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:26.334406 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:26.334423 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.335694 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.335698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:26.336958 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:26.336971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.337787 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.337791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:26.337984 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:26.338592 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:26.338595 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:26.338632 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:26.338700 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:26.339047 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:26.339052 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:26.339057 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:14:26.339178 (+ 121us) client_negotiation.cc:240] Negotiation successful
0812 07:14:26.339436 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":256,"threads_started":1}
I20260812 07:14:26.341779 24447 heartbeater.cc:347] Connected to a master server at 127.21.181.126:40809
I20260812 07:14:26.342070 24447 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:26.342881 24447 heartbeater.cc:511] Master 127.21.181.126:40809 requested a full tablet report, sending...
I20260812 07:14:26.344940 24126 ts_manager.cc:194] Registered new tserver with Master: 5fa4d71171a94a51913ef98893604b2e (127.21.181.65:36007)
I20260812 07:14:26.346421 24126 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.65:58947
W20260812 07:14:26.656263 24455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:26.656505 24455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:26.656595 24455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:26.663543 24455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:26.663632 24455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:26.663738 24455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:14:26.672209 24455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:40809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:26.673250 24455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:26.674923 24455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:26.686894 24464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:26.686947 24467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:26.687069 24465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:26.688127 24455 server_base.cc:1034] running on GCE node
I20260812 07:14:26.688699 24455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:26.689544 24455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:26.690743 24455 hybrid_clock.cc:648] HybridClock initialized: now 1786518866690688 us; error 47 us; skew 500 ppm
I20260812 07:14:26.691036 24455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:26.693163 24455 webserver.cc:460] Webserver started at http://127.21.181.66:38453/ using document root <none> and password file <none>
I20260812 07:14:26.693823 24455 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:26.693919 24455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:26.694206 24455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:26.696949 24455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "3fd71b67411c44beb0fd4dbecf2eba60"
format_stamp: "Formatted at 2026-08-12 07:14:26 on dist-test-slave-lgr5"
server_key: "46194ce10d6bb859afea552bff61d90a"
server_key_iv: "d5d69b496d183cd5d269012bd61a88ea"
server_key_version: "encryptionkey@0"
I20260812 07:14:26.697696 24455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "3fd71b67411c44beb0fd4dbecf2eba60"
format_stamp: "Formatted at 2026-08-12 07:14:26 on dist-test-slave-lgr5"
server_key: "46194ce10d6bb859afea552bff61d90a"
server_key_iv: "d5d69b496d183cd5d269012bd61a88ea"
server_key_version: "encryptionkey@0"
I20260812 07:14:26.702349 24455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:26.705538 24484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:26.706707 24455 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:14:26.706851 24455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "3fd71b67411c44beb0fd4dbecf2eba60"
format_stamp: "Formatted at 2026-08-12 07:14:26 on dist-test-slave-lgr5"
server_key: "46194ce10d6bb859afea552bff61d90a"
server_key_iv: "d5d69b496d183cd5d269012bd61a88ea"
server_key_version: "encryptionkey@0"
I20260812 07:14:26.707032 24455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:26.745430 24455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:26.746286 24455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:26.746575 24455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:26.747428 24455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:26.748792 24455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:26.748888 24455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:26.748991 24455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:26.749066 24455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:26.778257 24455 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:36085
I20260812 07:14:26.778313 24693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:36085 every 8 connection(s)
I20260812 07:14:26.779455 24455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:14:26.785007 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24455
I20260812 07:14:26.785166 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:14:26.785561 22229 external_mini_cluster.cc:1428] Setting key 6c3366cb2741927385c07f01d54bf320
I20260812 07:14:26.788188 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:40809
--builtin_ntp_servers=127.21.181.84:36565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:14:26.792021 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:26.781795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:60637 (local address 127.21.181.126:40809)
0812 07:14:26.781965 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:26.781970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:26.783247 (+ 1277us) server_negotiation.cc:407] Connection header received
0812 07:14:26.785064 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:26.785070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:26.785159 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:26.785247 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:26.786391 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.787127 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.789037 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.789654 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.789788 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:26.791254 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:26.791261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:26.791265 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:26.791289 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:26.791327 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:26.791341 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:26.791423 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:26.791645 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 07:14:26.791736 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:14:26.793198 24700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:26.782264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:40809 (local address 127.21.181.66:60637)
0812 07:14:26.783105 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:26.783147 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:26.784802 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:26.785493 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:26.785501 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:26.785588 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:26.786213 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:26.786239 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.787277 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.787281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:26.788881 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:26.788898 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:26.789898 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:26.789906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:26.790190 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:26.790899 (+ 709us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:26.790903 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:26.790953 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:26.791057 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:26.791441 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:26.791451 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:26.791459 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:14:26.791622 (+ 163us) client_negotiation.cc:240] Negotiation successful
0812 07:14:26.791972 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":262,"threads_started":1}
I20260812 07:14:26.794499 24696 heartbeater.cc:347] Connected to a master server at 127.21.181.126:40809
I20260812 07:14:26.794780 24696 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:26.795593 24696 heartbeater.cc:511] Master 127.21.181.126:40809 requested a full tablet report, sending...
I20260812 07:14:26.797097 24126 ts_manager.cc:194] Registered new tserver with Master: 3fd71b67411c44beb0fd4dbecf2eba60 (127.21.181.66:36085)
I20260812 07:14:26.798086 24126 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.66:60637
W20260812 07:14:27.109998 24703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:27.110241 24703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:27.110361 24703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:27.117424 24703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:27.117515 24703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:27.117654 24703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:14:27.126215 24703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:40809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:27.127326 24703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:27.129076 24703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:27.141017 24716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:27.141156 24714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:27.141156 24713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:27.142100 24703 server_base.cc:1034] running on GCE node
I20260812 07:14:27.142671 24703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:27.143491 24703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:27.144685 24703 hybrid_clock.cc:648] HybridClock initialized: now 1786518867144629 us; error 43 us; skew 500 ppm
I20260812 07:14:27.144985 24703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:27.147054 24703 webserver.cc:460] Webserver started at http://127.21.181.67:36077/ using document root <none> and password file <none>
I20260812 07:14:27.147675 24703 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:27.147773 24703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:27.148068 24703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:27.150772 24703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "93913ec14f954f908bc6d5ebe6fdc4cc"
format_stamp: "Formatted at 2026-08-12 07:14:27 on dist-test-slave-lgr5"
server_key: "4584af7154c07291d8e85b2a753a4c18"
server_key_iv: "773613e3089f5938b9aa953c3ad606d2"
server_key_version: "encryptionkey@0"
I20260812 07:14:27.151548 24703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "93913ec14f954f908bc6d5ebe6fdc4cc"
format_stamp: "Formatted at 2026-08-12 07:14:27 on dist-test-slave-lgr5"
server_key: "4584af7154c07291d8e85b2a753a4c18"
server_key_iv: "773613e3089f5938b9aa953c3ad606d2"
server_key_version: "encryptionkey@0"
I20260812 07:14:27.156338 24703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260812 07:14:27.159670 24722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:27.160959 24703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260812 07:14:27.161109 24703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "93913ec14f954f908bc6d5ebe6fdc4cc"
format_stamp: "Formatted at 2026-08-12 07:14:27 on dist-test-slave-lgr5"
server_key: "4584af7154c07291d8e85b2a753a4c18"
server_key_iv: "773613e3089f5938b9aa953c3ad606d2"
server_key_version: "encryptionkey@0"
I20260812 07:14:27.161312 24703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:27.181546 24703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:27.182278 24703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:27.182559 24703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:27.183468 24703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:27.184845 24703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:27.184947 24703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:27.185057 24703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:27.185133 24703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:27.222199 24703 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:40091
I20260812 07:14:27.222266 24920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:40091 every 8 connection(s)
I20260812 07:14:27.223423 24703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:14:27.230922 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 24703
I20260812 07:14:27.231088 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:14:27.231570 22229 external_mini_cluster.cc:1428] Setting key 6fae855b7eea58bbf2c271005f106632
I20260812 07:14:27.235410 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.225780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:50677 (local address 127.21.181.126:40809)
0812 07:14:27.225937 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.225942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.227252 (+ 1310us) server_negotiation.cc:407] Connection header received
0812 07:14:27.229006 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.229012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.229092 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.229181 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:27.230204 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.231157 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.232906 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.233402 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.233541 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.234787 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:27.234792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:27.234795 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:27.234812 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:27.234846 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:27.234857 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:27.234923 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.235111 (+ 188us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.235186 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:14:27.236342 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.226240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:40809 (local address 127.21.181.67:50677)
0812 07:14:27.227099 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:27.227143 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:27.228735 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:27.229410 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:27.229418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:27.229508 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:27.230034 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.230051 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.231323 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.231328 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.232773 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.232790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.233605 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.233610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.233800 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.234505 (+ 705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:27.234507 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:27.234546 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:27.234619 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:27.234930 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:27.234936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:27.234941 (+ 5us) client_negotiation.cc:769] Sending connection context
0812 07:14:27.235098 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 07:14:27.235358 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":264,"threads_started":1}
I20260812 07:14:27.237555 24921 heartbeater.cc:347] Connected to a master server at 127.21.181.126:40809
I20260812 07:14:27.237804 24921 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:27.238457 24921 heartbeater.cc:511] Master 127.21.181.126:40809 requested a full tablet report, sending...
I20260812 07:14:27.239661 24126 ts_manager.cc:194] Registered new tserver with Master: 93913ec14f954f908bc6d5ebe6fdc4cc (127.21.181.67:40091)
I20260812 07:14:27.240504 24126 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.67:50677
I20260812 07:14:27.246203 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:14:27.255637 24449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.250036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.21.181.126:40809)
0812 07:14:27.250190 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.250195 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.250602 (+ 407us) server_negotiation.cc:407] Connection header received
0812 07:14:27.250746 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.250751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.250825 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.250915 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:27.251385 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.252042 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.253374 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.253775 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.253892 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.254951 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:27.254956 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:27.254959 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:27.254975 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:27.255005 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:27.255016 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:27.255153 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.255335 (+ 182us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.255388 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:14:27.259011 24126 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:14:27.265504 24126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:14:27.289779 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.276709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54624 (local address 127.21.181.66:36085)
0812 07:14:27.277279 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.277288 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.278310 (+ 1022us) server_negotiation.cc:407] Connection header received
0812 07:14:27.278753 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.278762 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.279075 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.279388 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:27.281336 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.283358 (+ 2022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.286890 (+ 3532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.288235 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.288357 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.288646 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.289261 (+ 615us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.289427 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":278,"threads_started":1}
I20260812 07:14:27.289820 24943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.276375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36085 (local address 127.0.0.1:54624)
0812 07:14:27.278153 (+ 1778us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:27.278205 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:27.278455 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:27.279813 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:27.279822 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:27.279853 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:27.280386 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.280400 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.283550 (+ 3150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.283556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.286710 (+ 3154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.286724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.288766 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.288772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.289018 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.289079 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 07:14:27.289198 (+ 119us) client_negotiation.cc:240] Negotiation successful
0812 07:14:27.289383 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1527,"mutex_wait_us":24,"thread_start_us":1763,"threads_started":1}
I20260812 07:14:27.289896 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.276859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.21.181.65:36007)
0812 07:14:27.277694 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.277711 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.279578 (+ 1867us) server_negotiation.cc:407] Connection header received
0812 07:14:27.279805 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.279814 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.280036 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.281314 (+ 1278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:27.281570 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.283140 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.287200 (+ 4060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.288483 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.288610 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.288843 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.289379 (+ 536us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.289551 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":1104,"threads_started":1}
I20260812 07:14:27.290686 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.277459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36007 (local address 127.0.0.1:53698)
0812 07:14:27.279102 (+ 1643us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:27.279121 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:27.279510 (+ 389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:27.280701 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:27.280706 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:27.280735 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:27.281168 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.281181 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.283310 (+ 2129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.283316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.287014 (+ 3698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.287027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.288975 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.288981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.289171 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.289202 (+ 31us) client_negotiation.cc:769] Sending connection context
0812 07:14:27.290077 (+ 875us) client_negotiation.cc:240] Negotiation successful
0812 07:14:27.290323 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1502,"mutex_wait_us":1259,"thread_start_us":1361,"threads_started":1}
I20260812 07:14:27.293919 24338 tablet_service.cc:1467] Processing CreateTablet for tablet 0e027d9284484b7cbba5652c31f0e40e (DEFAULT_TABLE table=test-table [id=f958be94e13c44beb6e715907eebfb2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:27.293916 24577 tablet_service.cc:1467] Processing CreateTablet for tablet 0e027d9284484b7cbba5652c31f0e40e (DEFAULT_TABLE table=test-table [id=f958be94e13c44beb6e715907eebfb2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:27.294680 24947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.278043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:40091 (local address 127.0.0.1:39348)
0812 07:14:27.286178 (+ 8135us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:27.286196 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:27.286321 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:27.287274 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:27.287278 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:27.287306 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:27.287650 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.287662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.289449 (+ 1787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.289454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.291531 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.291546 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.293275 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.293282 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.293445 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.293488 (+ 43us) client_negotiation.cc:769] Sending connection context
0812 07:14:27.293578 (+ 90us) client_negotiation.cc:240] Negotiation successful
0812 07:14:27.293688 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8008,"mutex_wait_us":1,"thread_start_us":7843,"threads_started":1}
I20260812 07:14:27.294798 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.278212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.21.181.67:40091)
0812 07:14:27.286849 (+ 8637us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.286855 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.286880 (+ 25us) server_negotiation.cc:407] Connection header received
0812 07:14:27.286949 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.286953 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.287100 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.287288 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:27.287834 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.289300 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.291771 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.293089 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.293228 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.293465 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.294260 (+ 795us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.294427 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8501,"thread_start_us":8359,"threads_started":1}
I20260812 07:14:27.295331 24338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e027d9284484b7cbba5652c31f0e40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:27.295394 24577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e027d9284484b7cbba5652c31f0e40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:27.298422 24812 tablet_service.cc:1467] Processing CreateTablet for tablet 0e027d9284484b7cbba5652c31f0e40e (DEFAULT_TABLE table=test-table [id=f958be94e13c44beb6e715907eebfb2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:27.299870 24812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e027d9284484b7cbba5652c31f0e40e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:27.306031 24951 tablet_bootstrap.cc:492] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Bootstrap starting.
I20260812 07:14:27.309530 24952 tablet_bootstrap.cc:492] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Bootstrap starting.
I20260812 07:14:27.309829 24951 tablet_bootstrap.cc:654] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:27.311188 24951 log.cc:824] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:27.312767 24952 tablet_bootstrap.cc:654] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:27.313167 24956 tablet_bootstrap.cc:492] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Bootstrap starting.
I20260812 07:14:27.313958 24952 log.cc:824] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:27.314270 24951 tablet_bootstrap.cc:492] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: No bootstrap required, opened a new log
I20260812 07:14:27.314558 24951 ts_tablet_manager.cc:1397] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260812 07:14:27.316305 24956 tablet_bootstrap.cc:654] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:27.317049 24952 tablet_bootstrap.cc:492] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: No bootstrap required, opened a new log
I20260812 07:14:27.317185 24956 log.cc:824] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:27.317301 24952 ts_tablet_manager.cc:1397] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260812 07:14:27.319396 24956 tablet_bootstrap.cc:492] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: No bootstrap required, opened a new log
I20260812 07:14:27.319617 24956 ts_tablet_manager.cc:1397] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:14:27.324677 24951 raft_consensus.cc:348] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.325073 24951 raft_consensus.cc:374] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:27.325170 24951 raft_consensus.cc:729] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fd71b67411c44beb0fd4dbecf2eba60, State: Initialized, Role: FOLLOWER
I20260812 07:14:27.325807 24951 consensus_queue.cc:260] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.327039 24956 raft_consensus.cc:348] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.327356 24956 raft_consensus.cc:374] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:27.327450 24956 raft_consensus.cc:729] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93913ec14f954f908bc6d5ebe6fdc4cc, State: Initialized, Role: FOLLOWER
I20260812 07:14:27.328024 24956 consensus_queue.cc:260] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.332736 24921 heartbeater.cc:503] Master 127.21.181.126:40809 was elected leader, sending a full tablet report...
I20260812 07:14:27.333403 24956 ts_tablet_manager.cc:1428] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260812 07:14:27.333210 24952 raft_consensus.cc:348] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.333604 24952 raft_consensus.cc:374] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:27.333760 24952 raft_consensus.cc:729] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fa4d71171a94a51913ef98893604b2e, State: Initialized, Role: FOLLOWER
I20260812 07:14:27.334244 24696 heartbeater.cc:503] Master 127.21.181.126:40809 was elected leader, sending a full tablet report...
I20260812 07:14:27.335150 24951 ts_tablet_manager.cc:1428] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Time spent starting tablet: real 0.020s user 0.011s sys 0.006s
I20260812 07:14:27.336565 24952 consensus_queue.cc:260] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.339135 24447 heartbeater.cc:503] Master 127.21.181.126:40809 was elected leader, sending a full tablet report...
I20260812 07:14:27.339725 24952 ts_tablet_manager.cc:1428] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Time spent starting tablet: real 0.022s user 0.014s sys 0.000s
I20260812 07:14:27.360286 24963 raft_consensus.cc:482] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:14:27.360471 24963 raft_consensus.cc:504] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.361828 24963 leader_election.cc:288] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fd71b67411c44beb0fd4dbecf2eba60 (127.21.181.66:36085), 5fa4d71171a94a51913ef98893604b2e (127.21.181.65:36007)
I20260812 07:14:27.367220 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.362184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:57415 (local address 127.21.181.66:36085)
0812 07:14:27.362344 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.362350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.362492 (+ 142us) server_negotiation.cc:407] Connection header received
0812 07:14:27.362665 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.362669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.362721 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.362791 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:27.363222 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.363914 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.365617 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.366404 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.366543 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.366615 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.366877 (+ 262us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.366993 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:14:27.367221 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.362208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:36085 (local address 127.21.181.67:57415)
0812 07:14:27.362394 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:27.362412 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:27.362540 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:27.362890 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:27.362895 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:27.362928 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:27.363079 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.363087 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.364043 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.364048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.365438 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.365462 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.366545 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.366551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.366699 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.366723 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:14:27.366826 (+ 103us) client_negotiation.cc:240] Negotiation successful
0812 07:14:27.366958 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260812 07:14:27.367885 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.362296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:45573 (local address 127.21.181.65:36007)
0812 07:14:27.362458 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.362464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.363133 (+ 669us) server_negotiation.cc:407] Connection header received
0812 07:14:27.363297 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.363301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.363353 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.363424 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:27.363931 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.364633 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.366368 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.367113 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.367252 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.367317 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.367614 (+ 297us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.367705 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:14:27.368059 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.362229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36007 (local address 127.21.181.67:45573)
0812 07:14:27.363034 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:27.363048 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:27.363168 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:27.363455 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:27.363459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:27.363481 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:27.363796 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.363804 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.364775 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.364779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.366203 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:27.366215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.367277 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.367284 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:27.367440 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.367473 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:14:27.367568 (+ 95us) client_negotiation.cc:240] Negotiation successful
0812 07:14:27.367681 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"mutex_wait_us":146,"thread_start_us":347,"threads_started":1}
I20260812 07:14:27.368338 24611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e027d9284484b7cbba5652c31f0e40e" candidate_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" is_pre_election: true
I20260812 07:14:27.368758 24611 raft_consensus.cc:2454] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93913ec14f954f908bc6d5ebe6fdc4cc in term 0.
I20260812 07:14:27.368918 24368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e027d9284484b7cbba5652c31f0e40e" candidate_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa4d71171a94a51913ef98893604b2e" is_pre_election: true
I20260812 07:14:27.369374 24368 raft_consensus.cc:2454] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 93913ec14f954f908bc6d5ebe6fdc4cc in term 0.
I20260812 07:14:27.369385 24731 leader_election.cc:302] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd71b67411c44beb0fd4dbecf2eba60, 93913ec14f954f908bc6d5ebe6fdc4cc; no voters:
I20260812 07:14:27.369714 24963 raft_consensus.cc:2781] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:14:27.369867 24963 raft_consensus.cc:482] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:14:27.369976 24963 raft_consensus.cc:3037] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:27.372099 24963 raft_consensus.cc:504] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.372610 24963 leader_election.cc:288] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [CANDIDATE]: Term 1 election: Requested vote from peers 3fd71b67411c44beb0fd4dbecf2eba60 (127.21.181.66:36085), 5fa4d71171a94a51913ef98893604b2e (127.21.181.65:36007)
I20260812 07:14:27.373018 24611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e027d9284484b7cbba5652c31f0e40e" candidate_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fd71b67411c44beb0fd4dbecf2eba60"
I20260812 07:14:27.373078 24368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e027d9284484b7cbba5652c31f0e40e" candidate_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fa4d71171a94a51913ef98893604b2e"
I20260812 07:14:27.373226 24611 raft_consensus.cc:3037] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:27.373238 24368 raft_consensus.cc:3037] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:27.376338 24368 raft_consensus.cc:2454] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93913ec14f954f908bc6d5ebe6fdc4cc in term 1.
I20260812 07:14:27.376339 24611 raft_consensus.cc:2454] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 93913ec14f954f908bc6d5ebe6fdc4cc in term 1.
I20260812 07:14:27.376704 24731 leader_election.cc:302] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fd71b67411c44beb0fd4dbecf2eba60, 93913ec14f954f908bc6d5ebe6fdc4cc; no voters:
I20260812 07:14:27.376971 24963 raft_consensus.cc:2781] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:27.377383 24963 raft_consensus.cc:686] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 1 LEADER]: Becoming Leader. State: Replica: 93913ec14f954f908bc6d5ebe6fdc4cc, State: Running, Role: LEADER
I20260812 07:14:27.377858 24963 consensus_queue.cc:237] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } }
I20260812 07:14:27.382097 24126 catalog_manager.cc:5495] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 93913ec14f954f908bc6d5ebe6fdc4cc (127.21.181.67). New cstate: current_term: 1 leader_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "93913ec14f954f908bc6d5ebe6fdc4cc" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 40091 } health_report { overall_health: HEALTHY } } }
I20260812 07:14:27.409754 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.404838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39352 (local address 127.21.181.67:40091)
0812 07:14:27.405046 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.405053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.405101 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:14:27.405279 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.405283 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.405370 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.405481 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:14:27.405848 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.406534 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.407828 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.408260 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.408345 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.408773 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:14:27.409266 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:14:27.409392 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.409464 (+ 72us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.409501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:27.417555 24611 raft_consensus.cc:1261] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 1 FOLLOWER]: Refusing update from remote peer 93913ec14f954f908bc6d5ebe6fdc4cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:27.417577 24368 raft_consensus.cc:1261] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 1 FOLLOWER]: Refusing update from remote peer 93913ec14f954f908bc6d5ebe6fdc4cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:27.418398 24966 consensus_queue.cc:1035] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fa4d71171a94a51913ef98893604b2e" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:27.418720 24963 consensus_queue.cc:1035] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fd71b67411c44beb0fd4dbecf2eba60" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 36085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:27.428381 24971 mvcc.cc:205] Tried to move back new op lower bound from 7317581280932929536 to 7317581280783609856. Current Snapshot: MvccSnapshot[applied={T|T < 7317581280932929536}]
I20260812 07:14:27.440213 24126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:27.440438 24126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:27.443681 24126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 56a50246d85a4ad7bcd57d835dee5131: Sending DeleteTablet for 3 replicas of tablet 0e027d9284484b7cbba5652c31f0e40e
I20260812 07:14:27.444869 24338 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e027d9284484b7cbba5652c31f0e40e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:27 UTC) from {username='slave'} at 127.0.0.1:53698
I20260812 07:14:27.444909 24577 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e027d9284484b7cbba5652c31f0e40e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:27 UTC) from {username='slave'} at 127.0.0.1:54624
I20260812 07:14:27.445528 24812 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e027d9284484b7cbba5652c31f0e40e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:27 UTC) from {username='slave'} at 127.0.0.1:39348
I20260812 07:14:27.446233 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24225
I20260812 07:14:27.454546 24997 tablet_replica.cc:330] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: stopping tablet replica
I20260812 07:14:27.455610 24997 raft_consensus.cc:2229] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:14:27.456858 24997 raft_consensus.cc:2258] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:27.459288 24996 tablet_replica.cc:330] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: stopping tablet replica
I20260812 07:14:27.459687 24996 raft_consensus.cc:2229] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:27.460439 24995 tablet_replica.cc:330] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: stopping tablet replica
I20260812 07:14:27.460807 24995 raft_consensus.cc:2229] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:27.460959 24997 ts_tablet_manager.cc:1905] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:27.461246 24995 raft_consensus.cc:2258] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:27.461714 24996 raft_consensus.cc:2258] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:27.463739 24995 ts_tablet_manager.cc:1905] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:27.463814 24996 ts_tablet_manager.cc:1905] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:27.468699 24997 ts_tablet_manager.cc:1918] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:27.468926 24997 log.cc:1196] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/0e027d9284484b7cbba5652c31f0e40e
I20260812 07:14:27.469528 24997 ts_tablet_manager.cc:1939] T 0e027d9284484b7cbba5652c31f0e40e P 93913ec14f954f908bc6d5ebe6fdc4cc: Deleting consensus metadata
I20260812 07:14:27.470443 24996 ts_tablet_manager.cc:1918] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:27.470688 24995 ts_tablet_manager.cc:1918] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:27.470883 24995 log.cc:1196] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/0e027d9284484b7cbba5652c31f0e40e
I20260812 07:14:27.471326 24100 catalog_manager.cc:4841] TS 93913ec14f954f908bc6d5ebe6fdc4cc (127.21.181.67:40091): tablet 0e027d9284484b7cbba5652c31f0e40e (table test-table [id=f958be94e13c44beb6e715907eebfb2f]) successfully deleted
I20260812 07:14:27.471457 24995 ts_tablet_manager.cc:1939] T 0e027d9284484b7cbba5652c31f0e40e P 3fd71b67411c44beb0fd4dbecf2eba60: Deleting consensus metadata
I20260812 07:14:27.472662 24996 log.cc:1196] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/0e027d9284484b7cbba5652c31f0e40e
I20260812 07:14:27.473145 24102 catalog_manager.cc:4841] TS 3fd71b67411c44beb0fd4dbecf2eba60 (127.21.181.66:36085): tablet 0e027d9284484b7cbba5652c31f0e40e (table test-table [id=f958be94e13c44beb6e715907eebfb2f]) successfully deleted
I20260812 07:14:27.473418 24996 ts_tablet_manager.cc:1939] T 0e027d9284484b7cbba5652c31f0e40e P 5fa4d71171a94a51913ef98893604b2e: Deleting consensus metadata
I20260812 07:14:27.474852 24103 catalog_manager.cc:4841] TS 5fa4d71171a94a51913ef98893604b2e (127.21.181.65:36007): tablet 0e027d9284484b7cbba5652c31f0e40e (table test-table [id=f958be94e13c44beb6e715907eebfb2f]) successfully deleted
I20260812 07:14:27.476142 24937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.447438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53706 (local address 127.21.181.65:36007)
0812 07:14:27.447609 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.447615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.459325 (+ 11710us) server_negotiation.cc:407] Connection header received
0812 07:14:27.459935 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.459940 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.460059 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.460158 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:27.460913 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.462043 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.472582 (+ 10539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.473214 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.473346 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.474744 (+ 1398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:27.474756 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:27.474773 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:27.475406 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:27.475452 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:27.475473 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:27.475555 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.475721 (+ 166us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.475809 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:14:27.476835 24441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:27.611672 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24225
I20260812 07:14:27.641007 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24455
I20260812 07:14:27.647667 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.641762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54640 (local address 127.21.181.66:36085)
0812 07:14:27.641976 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.641981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.642001 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:14:27.642137 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.642142 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.642241 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.642357 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:27.642765 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.643480 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.644767 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.645183 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.645291 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.646378 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:27.646388 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:27.646403 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:27.646960 (+ 557us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:27.647008 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:27.647028 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:27.647124 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.647257 (+ 133us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.647380 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260812 07:14:27.648366 24687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:27.735273 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24455
I20260812 07:14:27.760244 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24703
I20260812 07:14:27.767303 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:27.760985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.21.181.67:40091)
0812 07:14:27.761167 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:27.761173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:27.761279 (+ 106us) server_negotiation.cc:407] Connection header received
0812 07:14:27.761462 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:27.761466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:27.761543 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:27.761632 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:27.762040 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.762830 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.764072 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:27.764694 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:27.764809 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:27.765928 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:27.765940 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:27.765958 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:27.766575 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:27.766627 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:27.766650 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:27.766752 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:27.766881 (+ 129us) server_negotiation.cc:299] Negotiation successful
0812 07:14:27.766973 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:14:27.767915 24914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:27.900148 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24703
I20260812 07:14:27.929384 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 24068
I20260812 07:14:27.930488 24173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:28.039538 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 24068
2026-08-12T07:14:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2741 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:28 dist-test-slave-lgr5 krb5kdc[25010](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:28 dist-test-slave-lgr5 krb5kdc[25010](info): set up 2 sockets
Aug 12 07:14:28 dist-test-slave-lgr5 krb5kdc[25010](info): commencing operation
krb5kdc: starting...
W20260812 07:14:29.321578 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.007s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:29 dist-test-slave-lgr5 krb5kdc[25010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:14:29.601516 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:35375
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:35375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:29.943949 25036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:29.944262 25036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:29.944370 25036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:29.951366 25036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:29.951458 25036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:29.951511 25036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:29.951566 25036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:29.951609 25036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:29.961009 25036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:35375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:35375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:29.962157 25036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:29.963406 25036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:29.974462 25052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:29.974659 25054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:29.974642 25056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:29.975771 25036 server_base.cc:1034] running on GCE node
I20260812 07:14:29.976658 25036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:29.978359 25036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:29.979586 25036 hybrid_clock.cc:648] HybridClock initialized: now 1786518869979521 us; error 57 us; skew 500 ppm
I20260812 07:14:29.979985 25036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:29 dist-test-slave-lgr5 krb5kdc[25010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518869, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:29.983268 25036 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:14:29.984759 25036 webserver.cc:460] Webserver started at http://127.21.181.126:39911/ using document root <none> and password file <none>
I20260812 07:14:29.985405 25036 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:29.985517 25036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:29.985821 25036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:29.988777 25036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "2f29308be960429cb76e34bf65df567d"
format_stamp: "Formatted at 2026-08-12 07:14:29 on dist-test-slave-lgr5"
server_key: "fd0f7ce9bf44157ba94bb52ad768ef45"
server_key_iv: "25814a38961ee0d70430cd9966d7f2db"
server_key_version: "encryptionkey@0"
I20260812 07:14:29.989509 25036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "2f29308be960429cb76e34bf65df567d"
format_stamp: "Formatted at 2026-08-12 07:14:29 on dist-test-slave-lgr5"
server_key: "fd0f7ce9bf44157ba94bb52ad768ef45"
server_key_iv: "25814a38961ee0d70430cd9966d7f2db"
server_key_version: "encryptionkey@0"
I20260812 07:14:29.994436 25036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:29.997777 25067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:29.998991 25036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260812 07:14:29.999140 25036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "2f29308be960429cb76e34bf65df567d"
format_stamp: "Formatted at 2026-08-12 07:14:29 on dist-test-slave-lgr5"
server_key: "fd0f7ce9bf44157ba94bb52ad768ef45"
server_key_iv: "25814a38961ee0d70430cd9966d7f2db"
server_key_version: "encryptionkey@0"
I20260812 07:14:29.999315 25036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:30.017649 25036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:30.021320 25036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:30.021574 25036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:30.038450 25036 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:35375
I20260812 07:14:30.038472 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:35375 every 8 connection(s)
I20260812 07:14:30.039714 25036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:30.043190 25184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:30.046450 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25036
I20260812 07:14:30.046633 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:30.047214 22229 external_mini_cluster.cc:1428] Setting key d72556c3956e3f5183619f00fd42c56f
I20260812 07:14:30.055604 25184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d: Bootstrap starting.
I20260812 07:14:30.059197 25184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:30.060505 25184 log.cc:824] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:14:30 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:30.063463 25184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d: No bootstrap required, opened a new log
I20260812 07:14:30.071224 25188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:30.049125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.21.181.126:35375)
0812 07:14:30.049882 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:30.049897 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:30.049955 (+ 58us) server_negotiation.cc:407] Connection header received
0812 07:14:30.051749 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:30.051771 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:30.052192 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:30.052507 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:30.054240 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:30.055512 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:30.057295 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:30.057950 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:30.058075 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:30.062888 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:30.062896 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:30.062909 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:30.062963 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:30.066727 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:30.067453 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:30.067461 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:30.067469 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:30.067690 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:30.068050 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:30.068055 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:30.068059 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:30.068806 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:30.069109 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:30.069426 (+ 317us) server_negotiation.cc:299] Negotiation successful
0812 07:14:30.069883 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":372,"threads_started":1}
I20260812 07:14:30.073729 25184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f29308be960429cb76e34bf65df567d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 35375 } }
I20260812 07:14:30.074095 25184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:30.074209 25184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f29308be960429cb76e34bf65df567d, State: Initialized, Role: FOLLOWER
I20260812 07:14:30.074714 25184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f29308be960429cb76e34bf65df567d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 35375 } }
I20260812 07:14:30.074920 25184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:30.075026 25184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:30.075152 25184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:30.077446 25184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f29308be960429cb76e34bf65df567d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 35375 } }
I20260812 07:14:30.077879 25184 leader_election.cc:302] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f29308be960429cb76e34bf65df567d; no voters:
I20260812 07:14:30.078375 25184 leader_election.cc:288] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:30.078500 25193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:30.079134 25193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [term 1 LEADER]: Becoming Leader. State: Replica: 2f29308be960429cb76e34bf65df567d, State: Running, Role: LEADER
I20260812 07:14:30.079619 25193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f29308be960429cb76e34bf65df567d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 35375 } }
I20260812 07:14:30.080291 25184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:30.082823 25194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f29308be960429cb76e34bf65df567d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f29308be960429cb76e34bf65df567d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 35375 } } }
I20260812 07:14:30.082849 25195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f29308be960429cb76e34bf65df567d. Latest consensus state: current_term: 1 leader_uuid: "2f29308be960429cb76e34bf65df567d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f29308be960429cb76e34bf65df567d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 35375 } } }
I20260812 07:14:30.083288 25194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:30.083320 25195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:30.087775 25205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:30.091895 25205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:30.099368 25205 catalog_manager.cc:1347] Generated new cluster ID: 13e0d34e16b74a59a0804d778b7cc295
I20260812 07:14:30.099470 25205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:30.115868 25205 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:30.117690 25205 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:30.126199 25205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d: Generated new TSK 0
I20260812 07:14:30.127174 25205 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.21.181.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
Entry for principal kudu/127.21.181.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab.
I20260812 07:14:30.202543 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:35375
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:30.526904 25231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:30.527202 25231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:30.527350 25231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:30.534852 25231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:30.534968 25231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:30.535128 25231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:14:30.544593 25231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.65.keytab
--principal=kudu/127.21.181.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:35375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:30.545843 25231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:30.547101 25231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:30.559635 25243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:30.559679 25246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:30.559715 25244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:30.560613 25231 server_base.cc:1034] running on GCE node
I20260812 07:14:30.561173 25231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:30.562129 25231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:30.563324 25231 hybrid_clock.cc:648] HybridClock initialized: now 1786518870563272 us; error 46 us; skew 500 ppm
I20260812 07:14:30.563619 25231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:30 dist-test-slave-lgr5 krb5kdc[25010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:30.566654 25231 init.cc:377] Logged in from keytab as kudu/127.21.181.65@KRBTEST.COM (short username kudu)
I20260812 07:14:30.568050 25231 webserver.cc:460] Webserver started at http://127.21.181.65:41951/ using document root <none> and password file <none>
I20260812 07:14:30.568727 25231 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:30.568847 25231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:30.569140 25231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:30.572835 25231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "b059d581ca9240c6b39a6e6f71347467"
format_stamp: "Formatted at 2026-08-12 07:14:30 on dist-test-slave-lgr5"
server_key: "e61ff5435e77756b92a8ddf111fe7470"
server_key_iv: "675b42c38fb25566a247657675d79741"
server_key_version: "encryptionkey@0"
I20260812 07:14:30.573590 25231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "b059d581ca9240c6b39a6e6f71347467"
format_stamp: "Formatted at 2026-08-12 07:14:30 on dist-test-slave-lgr5"
server_key: "e61ff5435e77756b92a8ddf111fe7470"
server_key_iv: "675b42c38fb25566a247657675d79741"
server_key_version: "encryptionkey@0"
I20260812 07:14:30.578610 25231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:14:30.582051 25257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:30.583292 25231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:14:30.583456 25231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "b059d581ca9240c6b39a6e6f71347467"
format_stamp: "Formatted at 2026-08-12 07:14:30 on dist-test-slave-lgr5"
server_key: "e61ff5435e77756b92a8ddf111fe7470"
server_key_iv: "675b42c38fb25566a247657675d79741"
server_key_version: "encryptionkey@0"
I20260812 07:14:30.583631 25231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:30.620604 25231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:30.624960 25231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:30.625236 25231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:30.626295 25231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:30.627645 25231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:30.627730 25231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:30.627820 25231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:30.627864 25231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:30.657449 25231 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:34743
I20260812 07:14:30.657503 25452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:34743 every 8 connection(s)
I20260812 07:14:30.658710 25231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:14:30.661460 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25231
I20260812 07:14:30.661629 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:14:30.662154 22229 external_mini_cluster.cc:1428] Setting key cc35df69745d5f41b882f7db3bd45e5a
Aug 12 07:14:30 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518870, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.65@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:30.679394 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:30.661098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:54477 (local address 127.21.181.126:35375)
0812 07:14:30.661585 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:30.661591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:30.662606 (+ 1015us) server_negotiation.cc:407] Connection header received
0812 07:14:30.664529 (+ 1923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:30.664533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:30.664593 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:30.664679 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:30.666574 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:30.667290 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:30.669271 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:30.669650 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:30.669744 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:30.673536 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:30.673541 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:30.673545 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:30.673576 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:30.676271 (+ 2695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:30.677060 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:30.677063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:30.677065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:30.677176 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:30.677592 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:30.677595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:30.677597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:30.677837 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:30.677956 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:30.678806 (+ 850us) server_negotiation.cc:299] Negotiation successful
0812 07:14:30.679053 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260812 07:14:30.680356 25463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:30.661610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:35375 (local address 127.21.181.65:54477)
0812 07:14:30.662408 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:30.662455 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:30.664237 (+ 1782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:30.664956 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:30.664967 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:30.665636 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:30.666399 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:30.666430 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:30.667440 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:30.667446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:30.669148 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:30.669162 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:30.669859 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:30.669866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:30.670117 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:30.670769 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:30.670796 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:30.673324 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:30.676401 (+ 3077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:30.676410 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:30.676426 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:30.676960 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:30.677264 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:30.677268 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:30.677271 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:30.677505 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:30.677959 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:30.677966 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:30.678359 (+ 393us) client_negotiation.cc:769] Sending connection context
0812 07:14:30.678653 (+ 294us) client_negotiation.cc:240] Negotiation successful
0812 07:14:30.679158 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":304,"threads_started":1}
I20260812 07:14:30.681881 25453 heartbeater.cc:347] Connected to a master server at 127.21.181.126:35375
I20260812 07:14:30.682192 25453 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:30.683003 25453 heartbeater.cc:511] Master 127.21.181.126:35375 requested a full tablet report, sending...
I20260812 07:14:30.685412 25098 ts_manager.cc:194] Registered new tserver with Master: b059d581ca9240c6b39a6e6f71347467 (127.21.181.65:34743)
I20260812 07:14:30.687331 25098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.65@KRBTEST.COM'} at 127.21.181.65:54477
WARNING: no policy specified for kudu/127.21.181.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
Entry for principal kudu/127.21.181.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab.
I20260812 07:14:30.728816 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:35375
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:31.046969 25471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:31.047228 25471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:31.047367 25471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:31.055656 25471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:31.055768 25471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:31.055922 25471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:14:31.065096 25471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.66.keytab
--principal=kudu/127.21.181.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:35375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:31.066231 25471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:31.067444 25471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:31.080709 25480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:31.080681 25483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:31.080884 25490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:31.081790 25471 server_base.cc:1034] running on GCE node
I20260812 07:14:31.082381 25471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:31.083257 25471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:31.084455 25471 hybrid_clock.cc:648] HybridClock initialized: now 1786518871084401 us; error 43 us; skew 500 ppm
I20260812 07:14:31.084744 25471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:31 dist-test-slave-lgr5 krb5kdc[25010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518871, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:31.091311 25471 init.cc:377] Logged in from keytab as kudu/127.21.181.66@KRBTEST.COM (short username kudu)
I20260812 07:14:31.092864 25471 webserver.cc:460] Webserver started at http://127.21.181.66:40867/ using document root <none> and password file <none>
I20260812 07:14:31.093480 25471 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:31.093587 25471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:31.093847 25471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:31.096632 25471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "ce023342882a489c926af380afec8dc6"
format_stamp: "Formatted at 2026-08-12 07:14:31 on dist-test-slave-lgr5"
server_key: "a831969b7c550488da260335e6545a8c"
server_key_iv: "bb588321a5708bde969b3441f0ee4053"
server_key_version: "encryptionkey@0"
I20260812 07:14:31.097321 25471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "ce023342882a489c926af380afec8dc6"
format_stamp: "Formatted at 2026-08-12 07:14:31 on dist-test-slave-lgr5"
server_key: "a831969b7c550488da260335e6545a8c"
server_key_iv: "bb588321a5708bde969b3441f0ee4053"
server_key_version: "encryptionkey@0"
I20260812 07:14:31.102381 25471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 07:14:31.105630 25500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:31.106804 25471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:14:31.106937 25471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "ce023342882a489c926af380afec8dc6"
format_stamp: "Formatted at 2026-08-12 07:14:31 on dist-test-slave-lgr5"
server_key: "a831969b7c550488da260335e6545a8c"
server_key_iv: "bb588321a5708bde969b3441f0ee4053"
server_key_version: "encryptionkey@0"
I20260812 07:14:31.107116 25471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:31.124854 25471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:31.128685 25471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:31.128926 25471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:31.129765 25471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:31.130980 25471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:31.131057 25471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:31.131171 25471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:31.131225 25471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:31.160754 25471 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:38905
I20260812 07:14:31.160815 25713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:38905 every 8 connection(s)
I20260812 07:14:31.161947 25471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:14:31.172147 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25471
I20260812 07:14:31.172326 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:14:31.172873 22229 external_mini_cluster.cc:1428] Setting key 821bbcb1567f2ea2f00c291fcc7e70a6
Aug 12 07:14:31 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518871, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.66@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:31.180822 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.164308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:54739 (local address 127.21.181.126:35375)
0812 07:14:31.164482 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.164487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.165597 (+ 1110us) server_negotiation.cc:407] Connection header received
0812 07:14:31.167301 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.167306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.167379 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.167463 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:31.168902 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.169616 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.171064 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.171483 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.171578 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.175522 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:31.175527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:31.175530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:31.175560 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:31.177951 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.178627 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.178631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.178633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.178755 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.179167 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.179170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.179173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.179457 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:31.179591 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.180262 (+ 671us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.180490 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:14:31.181730 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.164757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:35375 (local address 127.21.181.66:54739)
0812 07:14:31.165424 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.165461 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.167060 (+ 1599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.167650 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.167658 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.168189 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:31.168759 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.168778 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.169732 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.169735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.170918 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.170954 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.171674 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.171679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.171946 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.172649 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:31.172684 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:31.175318 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:31.178086 (+ 2768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:31.178095 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:31.178109 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:31.178504 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:31.178857 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:31.178862 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:31.178866 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:31.179046 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:31.179580 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:31.179588 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:31.179879 (+ 291us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.180157 (+ 278us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.180658 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":256,"threads_started":1}
I20260812 07:14:31.182920 25716 heartbeater.cc:347] Connected to a master server at 127.21.181.126:35375
I20260812 07:14:31.183228 25716 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:31.184033 25716 heartbeater.cc:511] Master 127.21.181.126:35375 requested a full tablet report, sending...
I20260812 07:14:31.185567 25098 ts_manager.cc:194] Registered new tserver with Master: ce023342882a489c926af380afec8dc6 (127.21.181.66:38905)
I20260812 07:14:31.186408 25098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.66@KRBTEST.COM'} at 127.21.181.66:54739
WARNING: no policy specified for kudu/127.21.181.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
Entry for principal kudu/127.21.181.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab.
I20260812 07:14:31.234601 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:35375
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:44985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:31.555579 25727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:31.555825 25727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:31.555922 25727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:31.562784 25727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:31.562875 25727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:31.562986 25727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:14:31.572057 25727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:44985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.67.keytab
--principal=kudu/127.21.181.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:35375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:31.573182 25727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:31.574349 25727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:31.586812 25735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:31.586781 25738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:31.587414 25734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:31.587993 25727 server_base.cc:1034] running on GCE node
I20260812 07:14:31.588629 25727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:31.589479 25727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:31.590656 25727 hybrid_clock.cc:648] HybridClock initialized: now 1786518871590605 us; error 41 us; skew 500 ppm
I20260812 07:14:31.590940 25727 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:31 dist-test-slave-lgr5 krb5kdc[25010](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518871, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:31.593899 25727 init.cc:377] Logged in from keytab as kudu/127.21.181.67@KRBTEST.COM (short username kudu)
I20260812 07:14:31.595194 25727 webserver.cc:460] Webserver started at http://127.21.181.67:39235/ using document root <none> and password file <none>
I20260812 07:14:31.595811 25727 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:31.595925 25727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:31.596342 25727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:31.599126 25727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "8b669c36dc7c46eba230d2b091ca6f3f"
format_stamp: "Formatted at 2026-08-12 07:14:31 on dist-test-slave-lgr5"
server_key: "5a6d6522c612542987fe0708c8ea4369"
server_key_iv: "a6992031aa4f445464a68785d4e53fed"
server_key_version: "encryptionkey@0"
I20260812 07:14:31.599843 25727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "8b669c36dc7c46eba230d2b091ca6f3f"
format_stamp: "Formatted at 2026-08-12 07:14:31 on dist-test-slave-lgr5"
server_key: "5a6d6522c612542987fe0708c8ea4369"
server_key_iv: "a6992031aa4f445464a68785d4e53fed"
server_key_version: "encryptionkey@0"
I20260812 07:14:31.604692 25727 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:14:31.607869 25754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:31.609046 25727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:14:31.609192 25727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "8b669c36dc7c46eba230d2b091ca6f3f"
format_stamp: "Formatted at 2026-08-12 07:14:31 on dist-test-slave-lgr5"
server_key: "5a6d6522c612542987fe0708c8ea4369"
server_key_iv: "a6992031aa4f445464a68785d4e53fed"
server_key_version: "encryptionkey@0"
I20260812 07:14:31.609359 25727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:31.626761 25727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:31.630705 25727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:31.630995 25727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:31.631904 25727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:31.633217 25727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:31.633288 25727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:31.633402 25727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:31.633452 25727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:31.663235 25727 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:46633
I20260812 07:14:31.663305 25939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:46633 every 8 connection(s)
I20260812 07:14:31.664431 25727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:14:31.669766 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 25727
I20260812 07:14:31.669947 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:14:31.670507 22229 external_mini_cluster.cc:1428] Setting key 70474f08ec387e03add42d22e2c06943
Aug 12 07:14:31 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518871, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.67@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:14:31.682930 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.666800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:40915 (local address 127.21.181.126:35375)
0812 07:14:31.667002 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.667009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.668026 (+ 1017us) server_negotiation.cc:407] Connection header received
0812 07:14:31.669857 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.669863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.669949 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.670058 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:31.671808 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.672701 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.674285 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.674678 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.674775 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.677849 (+ 3074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:31.677855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:31.677859 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:31.677892 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:31.680388 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.681043 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.681047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.681049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.681165 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.681502 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.681505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.681507 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.681757 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:31.681890 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.682511 (+ 621us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.682672 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260812 07:14:31.683858 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.667213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:35375 (local address 127.21.181.67:40915)
0812 07:14:31.667880 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.667915 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.669553 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.670341 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.670353 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.671035 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:31.671648 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.671669 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.672850 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.672855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.674159 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.674171 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.674842 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.674847 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.675045 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.675610 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:14:31.675635 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:31.677655 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:31.680492 (+ 2837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:31.680499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:31.680512 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:31.680948 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:31.681249 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:14:31.681252 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:14:31.681254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:14:31.681425 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:14:31.681862 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:31.681868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:31.682195 (+ 327us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.682412 (+ 217us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.682763 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":262,"threads_started":1}
I20260812 07:14:31.685240 25940 heartbeater.cc:347] Connected to a master server at 127.21.181.126:35375
I20260812 07:14:31.685542 25940 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:31.686309 25940 heartbeater.cc:511] Master 127.21.181.126:35375 requested a full tablet report, sending...
I20260812 07:14:31.687667 25098 ts_manager.cc:194] Registered new tserver with Master: 8b669c36dc7c46eba230d2b091ca6f3f (127.21.181.67:46633)
I20260812 07:14:31.688570 25098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.21.181.67@KRBTEST.COM'} at 127.21.181.67:40915
I20260812 07:14:31.690639 25453 heartbeater.cc:503] Master 127.21.181.126:35375 was elected leader, sending a full tablet report...
I20260812 07:14:31.697494 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:14:31.712523 25456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.701286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.21.181.126:35375)
0812 07:14:31.701434 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.701438 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.701883 (+ 445us) server_negotiation.cc:407] Connection header received
0812 07:14:31.702010 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.702013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.702068 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.702141 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:31.703448 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.704216 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.705381 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.705772 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.705857 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.707904 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:31.707908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:31.707911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:31.707940 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:31.710124 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.710748 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.710751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.710754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.710877 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.711161 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.711164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.711166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.711488 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:31.711750 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.712021 (+ 271us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.712218 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:14:31.715768 25098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:14:31.722388 25098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:14:31.757278 25966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.734939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53754 (local address 127.21.181.67:46633)
0812 07:14:31.749477 (+ 14538us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.749486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.749520 (+ 34us) server_negotiation.cc:407] Connection header received
0812 07:14:31.749618 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.749625 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.749839 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.750036 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:31.750690 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.752199 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.754497 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.755861 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.755995 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.756246 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.756815 (+ 569us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.756968 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14376,"thread_start_us":14231,"threads_started":1}
I20260812 07:14:31.757323 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.733830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:46633 (local address 127.0.0.1:53754)
0812 07:14:31.748134 (+ 14304us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.748153 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.748261 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.750027 (+ 1766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.750031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.750054 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:31.750500 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.750515 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.752333 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.752338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.754343 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.754357 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.756407 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.756414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.756597 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.756658 (+ 61us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.756751 (+ 93us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.756940 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14173,"mutex_wait_us":84,"thread_start_us":11957,"threads_started":1}
I20260812 07:14:31.757347 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.733816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.21.181.66:38905)
0812 07:14:31.748505 (+ 14689us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.748513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.748543 (+ 30us) server_negotiation.cc:407] Connection header received
0812 07:14:31.748648 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.748656 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.748909 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.749867 (+ 958us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:31.750105 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.751443 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.754146 (+ 2703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.755758 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.755880 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.756119 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.756907 (+ 788us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.757059 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14531,"thread_start_us":14429,"threads_started":1}
I20260812 07:14:31.757972 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.733634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:38905 (local address 127.0.0.1:50782)
0812 07:14:31.746143 (+ 12509us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.746190 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.746347 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.749154 (+ 2807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.749162 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.749195 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:31.749660 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.749674 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.751596 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.751601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.753977 (+ 2376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.753993 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.756254 (+ 2261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.756264 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.756549 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.756630 (+ 81us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.756753 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.757611 (+ 858us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12281,"mutex_wait_us":750,"thread_start_us":14691,"threads_started":1}
I20260812 07:14:31.758563 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.734396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34743 (local address 127.0.0.1:38878)
0812 07:14:31.749629 (+ 15233us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.749644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.749760 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.750315 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.750319 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.750347 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:31.750766 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.750778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.752894 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.752898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.754498 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.754513 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.756328 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.756334 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.756530 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.756630 (+ 100us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.756753 (+ 123us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.758233 (+ 1480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15110,"mutex_wait_us":1365,"thread_start_us":14953,"threads_started":1}
I20260812 07:14:31.761099 25832 tablet_service.cc:1467] Processing CreateTablet for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e (DEFAULT_TABLE table=test-table [id=1d627719b6fd407ab7b138f756117335]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:31.761094 25589 tablet_service.cc:1467] Processing CreateTablet for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e (DEFAULT_TABLE table=test-table [id=1d627719b6fd407ab7b138f756117335]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:31.762226 25589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:31.762236 25832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:31.763839 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.735304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38878 (local address 127.21.181.65:34743)
0812 07:14:31.748996 (+ 13692us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.749006 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.749738 (+ 732us) server_negotiation.cc:407] Connection header received
0812 07:14:31.749889 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.749894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.750100 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.750280 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:31.750923 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.752768 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.754691 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.756122 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.756263 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.756465 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.756885 (+ 420us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.757059 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13519,"thread_start_us":13370,"threads_started":1}
I20260812 07:14:31.767304 25347 tablet_service.cc:1467] Processing CreateTablet for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e (DEFAULT_TABLE table=test-table [id=1d627719b6fd407ab7b138f756117335]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:31.768702 25347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:31.773238 25970 tablet_bootstrap.cc:492] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Bootstrap starting.
I20260812 07:14:31.775614 25971 tablet_bootstrap.cc:492] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Bootstrap starting.
I20260812 07:14:31.777437 25970 tablet_bootstrap.cc:654] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:31.778699 25970 log.cc:824] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:31.778894 25971 tablet_bootstrap.cc:654] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:31.780004 25971 log.cc:824] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:31.781695 25970 tablet_bootstrap.cc:492] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: No bootstrap required, opened a new log
I20260812 07:14:31.781955 25970 ts_tablet_manager.cc:1397] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260812 07:14:31.782424 25979 tablet_bootstrap.cc:492] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Bootstrap starting.
I20260812 07:14:31.783134 25971 tablet_bootstrap.cc:492] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: No bootstrap required, opened a new log
I20260812 07:14:31.783392 25971 ts_tablet_manager.cc:1397] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260812 07:14:31.785238 25979 tablet_bootstrap.cc:654] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:31.786350 25979 log.cc:824] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:31.788818 25979 tablet_bootstrap.cc:492] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: No bootstrap required, opened a new log
I20260812 07:14:31.789037 25979 ts_tablet_manager.cc:1397] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260812 07:14:31.792482 25970 raft_consensus.cc:348] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.792904 25970 raft_consensus.cc:374] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:31.793022 25970 raft_consensus.cc:729] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce023342882a489c926af380afec8dc6, State: Initialized, Role: FOLLOWER
I20260812 07:14:31.793309 25971 raft_consensus.cc:348] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.793730 25971 raft_consensus.cc:374] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:31.793839 25971 raft_consensus.cc:729] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b669c36dc7c46eba230d2b091ca6f3f, State: Initialized, Role: FOLLOWER
I20260812 07:14:31.793748 25970 consensus_queue.cc:260] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.794427 25971 consensus_queue.cc:260] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.794821 25716 heartbeater.cc:503] Master 127.21.181.126:35375 was elected leader, sending a full tablet report...
I20260812 07:14:31.795509 25940 heartbeater.cc:503] Master 127.21.181.126:35375 was elected leader, sending a full tablet report...
I20260812 07:14:31.796151 25971 ts_tablet_manager.cc:1428] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260812 07:14:31.797984 25979 raft_consensus.cc:348] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.798290 25979 raft_consensus.cc:374] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:31.798384 25979 raft_consensus.cc:729] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b059d581ca9240c6b39a6e6f71347467, State: Initialized, Role: FOLLOWER
I20260812 07:14:31.798449 25970 ts_tablet_manager.cc:1428] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260812 07:14:31.799094 25979 consensus_queue.cc:260] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.800782 25979 ts_tablet_manager.cc:1428] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Time spent starting tablet: real 0.012s user 0.009s sys 0.000s
I20260812 07:14:31.806732 25983 raft_consensus.cc:482] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:14:31.806934 25983 raft_consensus.cc:504] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.808285 25983 leader_election.cc:288] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b669c36dc7c46eba230d2b091ca6f3f (127.21.181.67:46633), b059d581ca9240c6b39a6e6f71347467 (127.21.181.65:34743)
I20260812 07:14:31.814956 25966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.808481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:34627 (local address 127.21.181.67:46633)
0812 07:14:31.808659 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.808666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.809667 (+ 1001us) server_negotiation.cc:407] Connection header received
0812 07:14:31.809861 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.809864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.809916 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.809995 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:31.810523 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.811308 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.813285 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.814089 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.814235 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.814310 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.814611 (+ 301us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.814731 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:14:31.815119 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.808418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:46633 (local address 127.21.181.66:34627)
0812 07:14:31.809552 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.809572 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.809728 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.810038 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.810043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.810068 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:31.810383 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.810392 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.811426 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.811431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.813140 (+ 1709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.813153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.814287 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.814293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.814465 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.814489 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.814574 (+ 85us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.814701 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"mutex_wait_us":25,"thread_start_us":824,"threads_started":1}
I20260812 07:14:31.815124 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.808886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:42463 (local address 127.21.181.65:34743)
0812 07:14:31.809064 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.809070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.809932 (+ 862us) server_negotiation.cc:407] Connection header received
0812 07:14:31.810073 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.810076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.810125 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.810192 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:31.810780 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.811477 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.813402 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.814272 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.814408 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.814476 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.814779 (+ 303us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.814894 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:14:31.815474 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.808652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:34743 (local address 127.21.181.66:42463)
0812 07:14:31.809841 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:31.809862 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:31.809962 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:31.810234 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:31.810239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:31.810265 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:31.810630 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.810641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.811615 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.811621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.813228 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:31.813241 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.814490 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.814497 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:31.814638 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.814661 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:14:31.814739 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:14:31.814859 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1077,"mutex_wait_us":17,"thread_start_us":906,"threads_started":1}
I20260812 07:14:31.815938 25865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a9595aef7c47e2a4cca5fcd3b4f05e" candidate_uuid: "ce023342882a489c926af380afec8dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" is_pre_election: true
I20260812 07:14:31.816398 25865 raft_consensus.cc:2454] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce023342882a489c926af380afec8dc6 in term 0.
I20260812 07:14:31.816318 25374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a9595aef7c47e2a4cca5fcd3b4f05e" candidate_uuid: "ce023342882a489c926af380afec8dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b059d581ca9240c6b39a6e6f71347467" is_pre_election: true
I20260812 07:14:31.816705 25374 raft_consensus.cc:2454] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce023342882a489c926af380afec8dc6 in term 0.
I20260812 07:14:31.816956 25502 leader_election.cc:302] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b669c36dc7c46eba230d2b091ca6f3f, ce023342882a489c926af380afec8dc6; no voters:
I20260812 07:14:31.817310 25983 raft_consensus.cc:2781] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:14:31.817435 25983 raft_consensus.cc:482] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:14:31.817543 25983 raft_consensus.cc:3037] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:31.819561 25983 raft_consensus.cc:504] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.820114 25983 leader_election.cc:288] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [CANDIDATE]: Term 1 election: Requested vote from peers 8b669c36dc7c46eba230d2b091ca6f3f (127.21.181.67:46633), b059d581ca9240c6b39a6e6f71347467 (127.21.181.65:34743)
I20260812 07:14:31.820428 25865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a9595aef7c47e2a4cca5fcd3b4f05e" candidate_uuid: "ce023342882a489c926af380afec8dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b669c36dc7c46eba230d2b091ca6f3f"
I20260812 07:14:31.820546 25374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32a9595aef7c47e2a4cca5fcd3b4f05e" candidate_uuid: "ce023342882a489c926af380afec8dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b059d581ca9240c6b39a6e6f71347467"
I20260812 07:14:31.820592 25865 raft_consensus.cc:3037] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:31.820725 25374 raft_consensus.cc:3037] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:31.822647 25865 raft_consensus.cc:2454] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce023342882a489c926af380afec8dc6 in term 1.
I20260812 07:14:31.822717 25374 raft_consensus.cc:2454] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce023342882a489c926af380afec8dc6 in term 1.
I20260812 07:14:31.822986 25502 leader_election.cc:302] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b669c36dc7c46eba230d2b091ca6f3f, ce023342882a489c926af380afec8dc6; no voters:
I20260812 07:14:31.823225 25983 raft_consensus.cc:2781] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:31.823629 25983 raft_consensus.cc:686] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 1 LEADER]: Becoming Leader. State: Replica: ce023342882a489c926af380afec8dc6, State: Running, Role: LEADER
I20260812 07:14:31.824040 25983 consensus_queue.cc:237] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } }
I20260812 07:14:31.828529 25096 catalog_manager.cc:5495] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce023342882a489c926af380afec8dc6 (127.21.181.66). New cstate: current_term: 1 leader_uuid: "ce023342882a489c926af380afec8dc6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce023342882a489c926af380afec8dc6" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 38905 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 } health_report { overall_health: UNKNOWN } } }
I20260812 07:14:31.855173 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.849910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.21.181.66:38905)
0812 07:14:31.850104 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.850109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.850206 (+ 97us) server_negotiation.cc:407] Connection header received
0812 07:14:31.850412 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.850415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.850501 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.850632 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:14:31.851042 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.851747 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.853166 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.853569 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.853675 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.854141 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:14:31.854600 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:14:31.854732 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.854831 (+ 99us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.854910 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:14:31.862821 25865 raft_consensus.cc:1261] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 1 FOLLOWER]: Refusing update from remote peer ce023342882a489c926af380afec8dc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:31.862843 25374 raft_consensus.cc:1261] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 1 FOLLOWER]: Refusing update from remote peer ce023342882a489c926af380afec8dc6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:31.863716 25995 consensus_queue.cc:1035] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b669c36dc7c46eba230d2b091ca6f3f" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:31.864068 25983 consensus_queue.cc:1035] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b059d581ca9240c6b39a6e6f71347467" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 34743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:31.886859 25098 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:31.887152 25098 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:31.891110 25098 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2f29308be960429cb76e34bf65df567d: Sending DeleteTablet for 3 replicas of tablet 32a9595aef7c47e2a4cca5fcd3b4f05e
I20260812 07:14:31.892246 25589 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:31 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:50782
I20260812 07:14:31.892271 25832 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:31 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:53754
I20260812 07:14:31.892292 25347 tablet_service.cc:1514] Processing DeleteTablet for tablet 32a9595aef7c47e2a4cca5fcd3b4f05e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:31 UTC) from {username='kudu', principal='kudu/127.21.181.126@KRBTEST.COM'} at 127.0.0.1:38878
I20260812 07:14:31.893198 26020 tablet_replica.cc:330] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: stopping tablet replica
I20260812 07:14:31.893738 26020 raft_consensus.cc:2229] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:31.894374 26020 raft_consensus.cc:2258] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:31.895349 26019 tablet_replica.cc:330] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: stopping tablet replica
I20260812 07:14:31.895447 26022 tablet_replica.cc:330] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: stopping tablet replica
I20260812 07:14:31.895876 26022 raft_consensus.cc:2229] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:31.895963 26019 raft_consensus.cc:2229] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:14:31.896366 26022 raft_consensus.cc:2258] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:31.897087 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25231
I20260812 07:14:31.897389 26020 ts_tablet_manager.cc:1905] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:31.897637 26019 raft_consensus.cc:2258] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:31.899834 26019 ts_tablet_manager.cc:1905] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:31.904624 26020 ts_tablet_manager.cc:1918] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:31.904785 26020 log.cc:1196] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/32a9595aef7c47e2a4cca5fcd3b4f05e
I20260812 07:14:31.905329 26020 ts_tablet_manager.cc:1939] T 32a9595aef7c47e2a4cca5fcd3b4f05e P 8b669c36dc7c46eba230d2b091ca6f3f: Deleting consensus metadata
I20260812 07:14:31.905357 26022 ts_tablet_manager.cc:1905] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:31.907110 25069 catalog_manager.cc:4841] TS 8b669c36dc7c46eba230d2b091ca6f3f (127.21.181.67:46633): tablet 32a9595aef7c47e2a4cca5fcd3b4f05e (table test-table [id=1d627719b6fd407ab7b138f756117335]) successfully deleted
I20260812 07:14:31.908454 26019 ts_tablet_manager.cc:1918] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:31.908697 26019 log.cc:1196] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/32a9595aef7c47e2a4cca5fcd3b4f05e
I20260812 07:14:31.909417 26019 ts_tablet_manager.cc:1939] T 32a9595aef7c47e2a4cca5fcd3b4f05e P ce023342882a489c926af380afec8dc6: Deleting consensus metadata
I20260812 07:14:31.911348 25074 catalog_manager.cc:4841] TS ce023342882a489c926af380afec8dc6 (127.21.181.66:38905): tablet 32a9595aef7c47e2a4cca5fcd3b4f05e (table test-table [id=1d627719b6fd407ab7b138f756117335]) successfully deleted
I20260812 07:14:31.912277 26022 ts_tablet_manager.cc:1918] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:31.912452 26022 log.cc:1196] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/32a9595aef7c47e2a4cca5fcd3b4f05e
I20260812 07:14:31.913074 26022 ts_tablet_manager.cc:1939] T 32a9595aef7c47e2a4cca5fcd3b4f05e P b059d581ca9240c6b39a6e6f71347467: Deleting consensus metadata
I20260812 07:14:31.914768 25068 catalog_manager.cc:4841] TS b059d581ca9240c6b39a6e6f71347467 (127.21.181.65:34743): tablet 32a9595aef7c47e2a4cca5fcd3b4f05e (table test-table [id=1d627719b6fd407ab7b138f756117335]) successfully deleted
Aug 12 07:14:31 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.65@KRBTEST.COM
I20260812 07:14:31.926134 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:31.898856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.21.181.65:34743)
0812 07:14:31.900434 (+ 1578us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:31.900441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:31.908858 (+ 8417us) server_negotiation.cc:407] Connection header received
0812 07:14:31.909381 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:31.909388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:31.909486 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:31.909580 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:31.912033 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.913076 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.914636 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:31.915284 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:31.915411 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:31.919771 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:31.919789 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:31.919807 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:31.919846 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:31.923300 (+ 3454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.923939 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.923949 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.923953 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.924141 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:31.924555 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:31.924560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:31.924564 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:31.925010 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:31.925307 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:31.925537 (+ 230us) server_negotiation.cc:299] Negotiation successful
0812 07:14:31.925750 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1447}
I20260812 07:14:31.926927 25447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:32.016736 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25231
I20260812 07:14:32.057394 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25471
Aug 12 07:14:32 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.66@KRBTEST.COM
I20260812 07:14:32.072791 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:32.058258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.21.181.66:38905)
0812 07:14:32.058456 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:32.058462 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:32.058489 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:14:32.058550 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:32.058554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:32.058640 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:32.058730 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:32.060026 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:32.060893 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:32.062135 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:32.062598 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:32.062708 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:32.067059 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:32.067073 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:32.067091 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:32.067138 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:32.070089 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:32.070722 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:32.070730 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:32.070732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:32.070883 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:32.071248 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:32.071252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:32.071254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:32.071618 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:32.071963 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:32.072217 (+ 254us) server_negotiation.cc:299] Negotiation successful
0812 07:14:32.072430 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:32.073530 25707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:32.183756 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25471
I20260812 07:14:32.209353 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25727
Aug 12 07:14:32 dist-test-slave-lgr5 krb5kdc[25010](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.67@KRBTEST.COM
I20260812 07:14:32.223219 25966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:32.210088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53766 (local address 127.21.181.67:46633)
0812 07:14:32.210285 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:32.210290 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:32.210309 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:14:32.210430 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:32.210434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:32.210512 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:32.210586 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:32.211790 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:32.212613 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:32.213925 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:32.214319 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:32.214410 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:32.218546 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:32.218560 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:32.218575 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:32.218606 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:32.221029 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:32.221600 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:32.221606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:32.221609 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:32.221740 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:32.222021 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:32.222024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:32.222026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:32.222337 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:32.222567 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:32.222753 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 07:14:32.222910 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260812 07:14:32.223838 25933 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:32.310077 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25727
I20260812 07:14:32.334132 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 25036
I20260812 07:14:32.335182 25170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:32.441295 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 25036
2026-08-12T07:14:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4401 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-12T07:14:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:32Z Disabled control of system clock
I20260812 07:14:32.529943 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:46787
--webserver_interface=127.21.181.126
--webserver_port=0
--builtin_ntp_servers=127.21.181.84:34809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:46787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:14:32.849341 26040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:32.849627 26040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:32.849753 26040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:32.856967 26040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:32.857071 26040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:32.857134 26040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:32.857210 26040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:32.857270 26040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:32.866537 26040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:46787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:46787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:32.867843 26040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:32.869683 26040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:32.880944 26048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:32.881012 26051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:32.881012 26049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:32.881907 26040 server_base.cc:1034] running on GCE node
I20260812 07:14:32.882639 26040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:32.883949 26040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:32.885154 26040 hybrid_clock.cc:648] HybridClock initialized: now 1786518872885088 us; error 53 us; skew 500 ppm
I20260812 07:14:32.885516 26040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:32.887590 26040 webserver.cc:460] Webserver started at http://127.21.181.126:35765/ using document root <none> and password file <none>
I20260812 07:14:32.888305 26040 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:32.888432 26040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:32.888732 26040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:32.891566 26040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "5004951800ce4c3e8309e4514eaa52b2"
format_stamp: "Formatted at 2026-08-12 07:14:32 on dist-test-slave-lgr5"
server_key: "79616d77ef27a2576a04eecfa8a3989a"
server_key_iv: "e5a8acf7ace80d1caac7cf95b5489835"
server_key_version: "encryptionkey@0"
I20260812 07:14:32.892395 26040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "5004951800ce4c3e8309e4514eaa52b2"
format_stamp: "Formatted at 2026-08-12 07:14:32 on dist-test-slave-lgr5"
server_key: "79616d77ef27a2576a04eecfa8a3989a"
server_key_iv: "e5a8acf7ace80d1caac7cf95b5489835"
server_key_version: "encryptionkey@0"
I20260812 07:14:32.897246 26040 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:14:32.900501 26060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:32.901734 26040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:14:32.901895 26040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "5004951800ce4c3e8309e4514eaa52b2"
format_stamp: "Formatted at 2026-08-12 07:14:32 on dist-test-slave-lgr5"
server_key: "79616d77ef27a2576a04eecfa8a3989a"
server_key_iv: "e5a8acf7ace80d1caac7cf95b5489835"
server_key_version: "encryptionkey@0"
I20260812 07:14:32.902089 26040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:32.923502 26040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:32.924278 26040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:32.924551 26040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:32.941460 26040 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:46787
I20260812 07:14:32.941468 26144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:46787 every 8 connection(s)
I20260812 07:14:32.942745 26040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:32.946374 26146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:32.952405 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26040
I20260812 07:14:32.952560 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:32.952992 22229 external_mini_cluster.cc:1428] Setting key 534b475dc50d887d402ec4e58289b2b0
I20260812 07:14:32.956943 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2: Bootstrap starting.
I20260812 07:14:32.959905 26146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:32.960886 26146 log.cc:824] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:32.963945 26146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2: No bootstrap required, opened a new log
I20260812 07:14:32.966064 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:32.954776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.21.181.126:46787)
0812 07:14:32.955734 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:32.955752 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:32.955792 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:14:32.957496 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:32.957518 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:32.957925 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:32.958268 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:32.958799 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:32.960070 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:32.961486 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:32.962241 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:32.962418 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:32.963419 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:32.963430 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:32.963446 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:32.964150 (+ 704us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:32.964253 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:32.964278 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:32.964418 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:32.964562 (+ 144us) server_negotiation.cc:299] Negotiation successful
0812 07:14:32.964866 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":338,"threads_started":1}
I20260812 07:14:32.972143 26146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5004951800ce4c3e8309e4514eaa52b2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46787 } }
I20260812 07:14:32.972455 26146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:32.972527 26146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5004951800ce4c3e8309e4514eaa52b2, State: Initialized, Role: FOLLOWER
I20260812 07:14:32.973002 26146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5004951800ce4c3e8309e4514eaa52b2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46787 } }
I20260812 07:14:32.973207 26146 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:32.973302 26146 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:32.973421 26146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:32.975688 26146 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5004951800ce4c3e8309e4514eaa52b2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46787 } }
I20260812 07:14:32.976171 26146 leader_election.cc:302] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5004951800ce4c3e8309e4514eaa52b2; no voters:
I20260812 07:14:32.976651 26146 leader_election.cc:288] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:32.976794 26154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:32.977423 26154 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [term 1 LEADER]: Becoming Leader. State: Replica: 5004951800ce4c3e8309e4514eaa52b2, State: Running, Role: LEADER
I20260812 07:14:32.977906 26154 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5004951800ce4c3e8309e4514eaa52b2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46787 } }
I20260812 07:14:32.978466 26146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:32.980733 26156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5004951800ce4c3e8309e4514eaa52b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5004951800ce4c3e8309e4514eaa52b2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46787 } } }
I20260812 07:14:32.981066 26156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:32.981431 26157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5004951800ce4c3e8309e4514eaa52b2. Latest consensus state: current_term: 1 leader_uuid: "5004951800ce4c3e8309e4514eaa52b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5004951800ce4c3e8309e4514eaa52b2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46787 } } }
I20260812 07:14:32.982081 26157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:32.983188 26162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:32.987910 26162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:32.996450 26162 catalog_manager.cc:1347] Generated new cluster ID: b670135db39d47c6829375fcc537e8e8
I20260812 07:14:32.996562 26162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:33.005309 26162 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:33.006577 26162 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:33.013350 26162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2: Generated new TSK 0
I20260812 07:14:33.014096 26162 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:33.022080 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.65:0
--local_ip_for_outbound_sockets=127.21.181.65
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:46787
--builtin_ntp_servers=127.21.181.84:34809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260812 07:14:33.367211 26180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:33.367453 26180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:33.367581 26180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:33.374497 26180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:33.374584 26180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:33.374727 26180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.65
I20260812 07:14:33.383467 26180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.181.65
--webserver_port=0
--tserver_master_addrs=127.21.181.126:46787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.65
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:33.384590 26180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:33.386204 26180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:33.398577 26194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:33.398653 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:33.398615 26201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:33.399808 26180 server_base.cc:1034] running on GCE node
I20260812 07:14:33.400410 26180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:33.401276 26180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:33.402475 26180 hybrid_clock.cc:648] HybridClock initialized: now 1786518873402431 us; error 32 us; skew 500 ppm
I20260812 07:14:33.402760 26180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:33.404771 26180 webserver.cc:460] Webserver started at http://127.21.181.65:35871/ using document root <none> and password file <none>
I20260812 07:14:33.405483 26180 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:33.405580 26180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:33.405828 26180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:33.408531 26180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data/instance:
uuid: "b04243a014eb4b92b615a29bc6366f69"
format_stamp: "Formatted at 2026-08-12 07:14:33 on dist-test-slave-lgr5"
server_key: "80e5ba254c060e21b23f75a5361af932"
server_key_iv: "51a482520316c9aa53ab7fd83df6a498"
server_key_version: "encryptionkey@0"
I20260812 07:14:33.409248 26180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance:
uuid: "b04243a014eb4b92b615a29bc6366f69"
format_stamp: "Formatted at 2026-08-12 07:14:33 on dist-test-slave-lgr5"
server_key: "80e5ba254c060e21b23f75a5361af932"
server_key_iv: "51a482520316c9aa53ab7fd83df6a498"
server_key_version: "encryptionkey@0"
I20260812 07:14:33.413825 26180 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:14:33.417021 26215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:33.418365 26180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260812 07:14:33.418491 26180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal
uuid: "b04243a014eb4b92b615a29bc6366f69"
format_stamp: "Formatted at 2026-08-12 07:14:33 on dist-test-slave-lgr5"
server_key: "80e5ba254c060e21b23f75a5361af932"
server_key_iv: "51a482520316c9aa53ab7fd83df6a498"
server_key_version: "encryptionkey@0"
I20260812 07:14:33.418651 26180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:33.444587 26180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:33.445284 26180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:33.445521 26180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:33.446362 26180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:33.447707 26180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:33.447789 26180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:33.447868 26180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:33.447922 26180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:33.477195 26180 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.65:36637
I20260812 07:14:33.477243 26402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.65:36637 every 8 connection(s)
I20260812 07:14:33.478425 26180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/data/info.pb
I20260812 07:14:33.487733 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26180
I20260812 07:14:33.487875 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal/instance
I20260812 07:14:33.488371 22229 external_mini_cluster.cc:1428] Setting key aacf900f662c240b98155f8f1c30d318
I20260812 07:14:33.490397 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:33.480770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.65:59169 (local address 127.21.181.126:46787)
0812 07:14:33.481173 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:33.481177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:33.482219 (+ 1042us) server_negotiation.cc:407] Connection header received
0812 07:14:33.484185 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:33.484191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:33.484268 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:33.484374 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:33.485406 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.486103 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.487592 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.488054 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.488202 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:33.489552 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:33.489557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:33.489560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:33.489577 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:33.489619 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:33.489638 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:33.489710 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:33.489929 (+ 219us) server_negotiation.cc:299] Negotiation successful
0812 07:14:33.490034 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":201,"threads_started":1}
I20260812 07:14:33.491148 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.66:0
--local_ip_for_outbound_sockets=127.21.181.66
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:46787
--builtin_ntp_servers=127.21.181.84:34809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:14:33.491236 26411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:33.481222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:46787 (local address 127.21.181.65:59169)
0812 07:14:33.482061 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:33.482104 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:33.483911 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:33.484608 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:33.484615 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:33.484696 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:33.485234 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:33.485250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.486258 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.486262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:33.487452 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:33.487465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.488284 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.488292 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:33.488560 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:33.489220 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:33.489224 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:33.489280 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:33.489365 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:33.489724 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:33.489736 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:33.489744 (+ 8us) client_negotiation.cc:769] Sending connection context
0812 07:14:33.489908 (+ 164us) client_negotiation.cc:240] Negotiation successful
0812 07:14:33.490250 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":254,"threads_started":1}
I20260812 07:14:33.492559 26407 heartbeater.cc:347] Connected to a master server at 127.21.181.126:46787
I20260812 07:14:33.492853 26407 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:33.493643 26407 heartbeater.cc:511] Master 127.21.181.126:46787 requested a full tablet report, sending...
I20260812 07:14:33.495704 26087 ts_manager.cc:194] Registered new tserver with Master: b04243a014eb4b92b615a29bc6366f69 (127.21.181.65:36637)
I20260812 07:14:33.497490 26087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.65:59169
W20260812 07:14:33.811322 26412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:33.811583 26412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:33.811713 26412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:33.818753 26412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:33.818848 26412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:33.818972 26412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.66
I20260812 07:14:33.827680 26412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.21.181.66
--webserver_port=0
--tserver_master_addrs=127.21.181.126:46787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.66
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:33.828821 26412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:33.830528 26412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:33.843073 26420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:33.842898 26425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:33.842850 26423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:33.843888 26412 server_base.cc:1034] running on GCE node
I20260812 07:14:33.844498 26412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:33.845316 26412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:33.846527 26412 hybrid_clock.cc:648] HybridClock initialized: now 1786518873846470 us; error 51 us; skew 500 ppm
I20260812 07:14:33.846815 26412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:33.848862 26412 webserver.cc:460] Webserver started at http://127.21.181.66:34465/ using document root <none> and password file <none>
I20260812 07:14:33.849566 26412 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:33.849668 26412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:33.849954 26412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:33.852691 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data/instance:
uuid: "1a32fdd9471b4bfd937b5a7f2f861dea"
format_stamp: "Formatted at 2026-08-12 07:14:33 on dist-test-slave-lgr5"
server_key: "2bcffac21d80bf02b30316c7c3ca54cb"
server_key_iv: "22c85d0ac1d94e2663b288e880e98720"
server_key_version: "encryptionkey@0"
I20260812 07:14:33.853448 26412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance:
uuid: "1a32fdd9471b4bfd937b5a7f2f861dea"
format_stamp: "Formatted at 2026-08-12 07:14:33 on dist-test-slave-lgr5"
server_key: "2bcffac21d80bf02b30316c7c3ca54cb"
server_key_iv: "22c85d0ac1d94e2663b288e880e98720"
server_key_version: "encryptionkey@0"
I20260812 07:14:33.858176 26412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:33.861428 26433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:33.862617 26412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:14:33.862766 26412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal
uuid: "1a32fdd9471b4bfd937b5a7f2f861dea"
format_stamp: "Formatted at 2026-08-12 07:14:33 on dist-test-slave-lgr5"
server_key: "2bcffac21d80bf02b30316c7c3ca54cb"
server_key_iv: "22c85d0ac1d94e2663b288e880e98720"
server_key_version: "encryptionkey@0"
I20260812 07:14:33.862949 26412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:33.877363 26412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:33.878064 26412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:33.878304 26412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:33.879200 26412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:33.880640 26412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:33.880725 26412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:33.880803 26412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:33.880856 26412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:33.910336 26412 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.66:42259
I20260812 07:14:33.910388 26618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.66:42259 every 8 connection(s)
I20260812 07:14:33.911573 26412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/data/info.pb
I20260812 07:14:33.920548 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26412
I20260812 07:14:33.920732 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal/instance
I20260812 07:14:33.921309 22229 external_mini_cluster.cc:1428] Setting key 01e5d0e837aa952899293cede9e07ee1
I20260812 07:14:33.923391 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:33.913870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.66:34087 (local address 127.21.181.126:46787)
0812 07:14:33.914082 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:33.914089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:33.915420 (+ 1331us) server_negotiation.cc:407] Connection header received
0812 07:14:33.917187 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:33.917191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:33.917258 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:33.917344 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:33.918288 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.918988 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.920521 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.921103 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.921241 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:33.922562 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:33.922566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:33.922570 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:33.922586 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:33.922622 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:33.922638 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:33.922733 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:33.922993 (+ 260us) server_negotiation.cc:299] Negotiation successful
0812 07:14:33.923091 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:33.924042 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.181.67:0
--local_ip_for_outbound_sockets=127.21.181.67
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:46787
--builtin_ntp_servers=127.21.181.84:34809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260812 07:14:33.924819 26622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:33.914397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:46787 (local address 127.21.181.66:34087)
0812 07:14:33.915266 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:33.915312 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:33.916938 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:33.917521 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:33.917528 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:33.917609 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:33.918163 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:33.918180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.919106 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.919109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:33.920372 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:33.920385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:33.921329 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:33.921335 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:33.921561 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:33.922251 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:33.922254 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:33.922302 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:33.922403 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:33.922742 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:33.922752 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:33.922759 (+ 7us) client_negotiation.cc:769] Sending connection context
0812 07:14:33.923561 (+ 802us) client_negotiation.cc:240] Negotiation successful
0812 07:14:33.923831 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":259,"threads_started":1}
I20260812 07:14:33.926107 26619 heartbeater.cc:347] Connected to a master server at 127.21.181.126:46787
I20260812 07:14:33.926379 26619 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:33.927066 26619 heartbeater.cc:511] Master 127.21.181.126:46787 requested a full tablet report, sending...
I20260812 07:14:33.928462 26087 ts_manager.cc:194] Registered new tserver with Master: 1a32fdd9471b4bfd937b5a7f2f861dea (127.21.181.66:42259)
I20260812 07:14:33.929247 26087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.66:34087
W20260812 07:14:34.248186 26623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:34.248435 26623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:34.248515 26623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:34.255529 26623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:34.255615 26623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:34.255724 26623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.181.67
I20260812 07:14:34.264385 26623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:34809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.181.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.21.181.67
--webserver_port=0
--tserver_master_addrs=127.21.181.126:46787
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.181.67
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:34.265470 26623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:34.267140 26623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:34.279148 26634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:34.279188 26633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:34.279210 26640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:34.280337 26623 server_base.cc:1034] running on GCE node
I20260812 07:14:34.280885 26623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:34.281728 26623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:34.282943 26623 hybrid_clock.cc:648] HybridClock initialized: now 1786518874282897 us; error 47 us; skew 500 ppm
I20260812 07:14:34.283296 26623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260812 07:14:34.285619 26623 webserver.cc:460] Webserver started at http://127.21.181.67:39315/ using document root <none> and password file <none>
I20260812 07:14:34.286219 26623 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:34.286303 26623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:34.286558 26623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:34.289301 26623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data/instance:
uuid: "d41eb1eaf78546208ffe61ff6827a825"
format_stamp: "Formatted at 2026-08-12 07:14:34 on dist-test-slave-lgr5"
server_key: "88d83ec852b8ad3156eb9abcb638008f"
server_key_iv: "c2974ef43be5d348b5071252b5ed6e91"
server_key_version: "encryptionkey@0"
I20260812 07:14:34.290107 26623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance:
uuid: "d41eb1eaf78546208ffe61ff6827a825"
format_stamp: "Formatted at 2026-08-12 07:14:34 on dist-test-slave-lgr5"
server_key: "88d83ec852b8ad3156eb9abcb638008f"
server_key_iv: "c2974ef43be5d348b5071252b5ed6e91"
server_key_version: "encryptionkey@0"
I20260812 07:14:34.294808 26623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:14:34.298372 26655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:34.299561 26623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:14:34.299683 26623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal
uuid: "d41eb1eaf78546208ffe61ff6827a825"
format_stamp: "Formatted at 2026-08-12 07:14:34 on dist-test-slave-lgr5"
server_key: "88d83ec852b8ad3156eb9abcb638008f"
server_key_iv: "c2974ef43be5d348b5071252b5ed6e91"
server_key_version: "encryptionkey@0"
I20260812 07:14:34.299844 26623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:34.322187 26623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:34.322842 26623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:34.323086 26623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:34.323951 26623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260812 07:14:34.325301 26623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 07:14:34.325385 26623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:34.325459 26623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260812 07:14:34.325510 26623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:34.354252 26623 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.67:33709
I20260812 07:14:34.354337 26848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.67:33709 every 8 connection(s)
I20260812 07:14:34.355399 26623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/data/info.pb
I20260812 07:14:34.362677 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26623
I20260812 07:14:34.362846 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal/instance
I20260812 07:14:34.363294 22229 external_mini_cluster.cc:1428] Setting key a2f214e27892871b7cc1b0969c122aa5
I20260812 07:14:34.367403 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.357697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:44237 (local address 127.21.181.126:46787)
0812 07:14:34.357883 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.357887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.359124 (+ 1237us) server_negotiation.cc:407] Connection header received
0812 07:14:34.360856 (+ 1732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.360860 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.360926 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.361012 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:34.361931 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.362846 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.364665 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.365208 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.365303 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.366747 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:34.366754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:34.366759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:34.366781 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:34.366843 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:34.366861 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:34.366931 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.367094 (+ 163us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.367170 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:14:34.368286 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.358162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:46787 (local address 127.21.181.67:44237)
0812 07:14:34.358978 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:34.359021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:34.360617 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:34.361195 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:34.361202 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:34.361287 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:14:34.361805 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.361822 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.363001 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.363006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.364511 (+ 1505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.364526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.365418 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.365425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.365747 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.366435 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0812 07:14:34.366438 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:14:34.366474 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0812 07:14:34.366557 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:14:34.366933 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:14:34.366941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:14:34.366947 (+ 6us) client_negotiation.cc:769] Sending connection context
0812 07:14:34.367080 (+ 133us) client_negotiation.cc:240] Negotiation successful
0812 07:14:34.367327 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":257,"threads_started":1}
I20260812 07:14:34.369457 26849 heartbeater.cc:347] Connected to a master server at 127.21.181.126:46787
I20260812 07:14:34.369711 26849 heartbeater.cc:464] Registering TS with master...
I20260812 07:14:34.370370 26849 heartbeater.cc:511] Master 127.21.181.126:46787 requested a full tablet report, sending...
I20260812 07:14:34.371619 26087 ts_manager.cc:194] Registered new tserver with Master: d41eb1eaf78546208ffe61ff6827a825 (127.21.181.67:33709)
I20260812 07:14:34.372407 26087 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.21.181.67:44237
I20260812 07:14:34.377851 22229 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260812 07:14:34.387058 26410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.381624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43290 (local address 127.21.181.126:46787)
0812 07:14:34.381802 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.381808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.382180 (+ 372us) server_negotiation.cc:407] Connection header received
0812 07:14:34.382345 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.382351 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.382424 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.382514 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:34.382982 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.383637 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.385010 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.385387 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.385502 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.386430 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:34.386436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:34.386439 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:34.386456 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:34.386492 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:34.386504 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:34.386575 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.386705 (+ 130us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.386786 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260812 07:14:34.390375 26087 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260812 07:14:34.397017 26087 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 07:14:34.431638 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.407914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:37938)
0812 07:14:34.420737 (+ 12823us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:34.420755 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:34.420869 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:34.422081 (+ 1212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:34.422089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:34.422124 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:34.423650 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.423664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.426457 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.426462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.428549 (+ 2087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.428562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.430357 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.430363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.430904 (+ 541us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.430957 (+ 53us) client_negotiation.cc:769] Sending connection context
0812 07:14:34.431092 (+ 135us) client_negotiation.cc:240] Negotiation successful
0812 07:14:34.431281 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12682,"mutex_wait_us":29,"thread_start_us":12504,"threads_started":1}
I20260812 07:14:34.432453 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.408944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.21.181.65:36637)
0812 07:14:34.421387 (+ 12443us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.421397 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.421430 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:14:34.421558 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.421567 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.421814 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.423872 (+ 2058us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:34.424164 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.426309 (+ 2145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.428741 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.430148 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.430290 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.430530 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.431914 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.432114 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12271,"thread_start_us":12147,"threads_started":1}
I20260812 07:14:34.433709 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.408365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51730 (local address 127.21.181.67:33709)
0812 07:14:34.425743 (+ 17378us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.425750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.425785 (+ 35us) server_negotiation.cc:407] Connection header received
0812 07:14:34.425878 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.425885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.426105 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.426297 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:34.427376 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.428953 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.431503 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.432659 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.432794 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.433059 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.433185 (+ 126us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.433337 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17232,"thread_start_us":17079,"threads_started":1}
I20260812 07:14:34.435735 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.407596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.67:33709 (local address 127.0.0.1:51730)
0812 07:14:34.419750 (+ 12154us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:34.419784 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:34.419954 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:34.426602 (+ 6648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:34.426607 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:34.426634 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:34.427123 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.427141 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.429129 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.429134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.431313 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.431327 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.432846 (+ 1519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.432852 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.433031 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.433063 (+ 32us) client_negotiation.cc:769] Sending connection context
0812 07:14:34.433145 (+ 82us) client_negotiation.cc:240] Negotiation successful
0812 07:14:34.433264 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11927,"mutex_wait_us":70,"thread_start_us":18305,"threads_started":1}
I20260812 07:14:34.436374 26871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.408293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:42259 (local address 127.0.0.1:59834)
0812 07:14:34.426184 (+ 17891us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:34.426203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:34.426318 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:34.426993 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:34.427001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:34.427027 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:34.427452 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.427464 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.429360 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.429365 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.431082 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.431095 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.435620 (+ 4525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.435630 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.435843 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.435876 (+ 33us) client_negotiation.cc:769] Sending connection context
0812 07:14:34.435967 (+ 91us) client_negotiation.cc:240] Negotiation successful
0812 07:14:34.436106 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17766,"thread_start_us":11099,"threads_started":1}
I20260812 07:14:34.435840 26283 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec63e2269184e4cb6474b704bd3c535 (DEFAULT_TABLE table=test-table [id=9d2b58cb56ef4d2581bcc017b67a29a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:34.437142 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.408646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.21.181.66:42259)
0812 07:14:34.425292 (+ 16646us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.425303 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.426448 (+ 1145us) server_negotiation.cc:407] Connection header received
0812 07:14:34.426564 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.426571 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.426776 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.426978 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:34.427623 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.429211 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.431292 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.432947 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.433069 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.433279 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.436640 (+ 3361us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.436800 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16478,"thread_start_us":16960,"threads_started":1}
I20260812 07:14:34.437398 26283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec63e2269184e4cb6474b704bd3c535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:34.439461 26742 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec63e2269184e4cb6474b704bd3c535 (DEFAULT_TABLE table=test-table [id=9d2b58cb56ef4d2581bcc017b67a29a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:34.440941 26742 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec63e2269184e4cb6474b704bd3c535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:34.440560 26518 tablet_service.cc:1467] Processing CreateTablet for tablet 7ec63e2269184e4cb6474b704bd3c535 (DEFAULT_TABLE table=test-table [id=9d2b58cb56ef4d2581bcc017b67a29a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 07:14:34.441854 26518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ec63e2269184e4cb6474b704bd3c535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:34.448266 26885 tablet_bootstrap.cc:492] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Bootstrap starting.
I20260812 07:14:34.451188 26885 tablet_bootstrap.cc:654] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:34.452134 26885 log.cc:824] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:34.454414 26885 tablet_bootstrap.cc:492] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: No bootstrap required, opened a new log
I20260812 07:14:34.454640 26885 ts_tablet_manager.cc:1397] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260812 07:14:34.455407 26888 tablet_bootstrap.cc:492] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Bootstrap starting.
I20260812 07:14:34.456769 26891 tablet_bootstrap.cc:492] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Bootstrap starting.
I20260812 07:14:34.458855 26888 tablet_bootstrap.cc:654] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:34.459738 26891 tablet_bootstrap.cc:654] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:34.459981 26888 log.cc:824] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:34.460712 26891 log.cc:824] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:34.462867 26888 tablet_bootstrap.cc:492] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: No bootstrap required, opened a new log
I20260812 07:14:34.463119 26888 ts_tablet_manager.cc:1397] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260812 07:14:34.464140 26885 raft_consensus.cc:348] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.464546 26885 raft_consensus.cc:374] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:34.464646 26885 raft_consensus.cc:729] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b04243a014eb4b92b615a29bc6366f69, State: Initialized, Role: FOLLOWER
I20260812 07:14:34.464874 26891 tablet_bootstrap.cc:492] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: No bootstrap required, opened a new log
I20260812 07:14:34.465113 26891 ts_tablet_manager.cc:1397] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260812 07:14:34.465214 26885 consensus_queue.cc:260] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.466497 26407 heartbeater.cc:503] Master 127.21.181.126:46787 was elected leader, sending a full tablet report...
I20260812 07:14:34.466881 26885 ts_tablet_manager.cc:1428] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Time spent starting tablet: real 0.012s user 0.009s sys 0.004s
I20260812 07:14:34.472997 26888 raft_consensus.cc:348] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.473408 26888 raft_consensus.cc:374] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:34.473528 26888 raft_consensus.cc:729] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a32fdd9471b4bfd937b5a7f2f861dea, State: Initialized, Role: FOLLOWER
I20260812 07:14:34.474092 26888 consensus_queue.cc:260] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.475093 26619 heartbeater.cc:503] Master 127.21.181.126:46787 was elected leader, sending a full tablet report...
I20260812 07:14:34.475711 26888 ts_tablet_manager.cc:1428] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260812 07:14:34.476531 26891 raft_consensus.cc:348] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.476881 26891 raft_consensus.cc:374] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:34.476971 26891 raft_consensus.cc:729] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41eb1eaf78546208ffe61ff6827a825, State: Initialized, Role: FOLLOWER
I20260812 07:14:34.477553 26891 consensus_queue.cc:260] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.478852 26849 heartbeater.cc:503] Master 127.21.181.126:46787 was elected leader, sending a full tablet report...
I20260812 07:14:34.479215 26891 ts_tablet_manager.cc:1428] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
W20260812 07:14:34.480387 26408 tablet.cc:2369] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:14:34.606912 26850 tablet.cc:2369] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 07:14:34.663558 26620 tablet.cc:2369] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:14:34.830991 26897 raft_consensus.cc:482] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 07:14:34.831286 26897 raft_consensus.cc:504] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.833024 26897 leader_election.cc:288] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b04243a014eb4b92b615a29bc6366f69 (127.21.181.65:36637), 1a32fdd9471b4bfd937b5a7f2f861dea (127.21.181.66:42259)
I20260812 07:14:34.838356 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.833569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:37925 (local address 127.21.181.66:42259)
0812 07:14:34.833750 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.833756 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.833774 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:14:34.833841 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.833845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.833912 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.833986 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:34.834439 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.835137 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.836805 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.837533 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.837677 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.837746 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.837978 (+ 232us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.838110 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:14:34.838376 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.833263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.66:42259 (local address 127.21.181.67:37925)
0812 07:14:34.833536 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:34.833557 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:34.833687 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:34.834048 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:34.834053 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:34.834086 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:34.834275 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.834292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.835277 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.835282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.836645 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.836664 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.837675 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.837681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.837836 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.837860 (+ 24us) client_negotiation.cc:769] Sending connection context
0812 07:14:34.837938 (+ 78us) client_negotiation.cc:240] Negotiation successful
0812 07:14:34.838069 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260812 07:14:34.838805 26898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.833351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.21.181.67:47165)
0812 07:14:34.833922 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:34.833935 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:14:34.834061 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:14:34.834383 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:14:34.834387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:14:34.834411 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0812 07:14:34.834702 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.834711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.835723 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.835728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.837044 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:14:34.837057 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.838119 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.838125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:14:34.838289 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.838312 (+ 23us) client_negotiation.cc:769] Sending connection context
0812 07:14:34.838395 (+ 83us) client_negotiation.cc:240] Negotiation successful
0812 07:14:34.838521 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":743,"threads_started":1}
I20260812 07:14:34.838795 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.833459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.21.181.67:47165 (local address 127.21.181.65:36637)
0812 07:14:34.833649 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.833654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.834033 (+ 379us) server_negotiation.cc:407] Connection header received
0812 07:14:34.834194 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.834198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.834269 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.834342 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0812 07:14:34.834860 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.835566 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.837220 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.837942 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.838109 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.838175 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.838436 (+ 261us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.838556 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:34.839354 26549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec63e2269184e4cb6474b704bd3c535" candidate_uuid: "d41eb1eaf78546208ffe61ff6827a825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" is_pre_election: true
I20260812 07:14:34.839969 26549 raft_consensus.cc:2454] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41eb1eaf78546208ffe61ff6827a825 in term 0.
I20260812 07:14:34.839905 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec63e2269184e4cb6474b704bd3c535" candidate_uuid: "d41eb1eaf78546208ffe61ff6827a825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04243a014eb4b92b615a29bc6366f69" is_pre_election: true
I20260812 07:14:34.840463 26318 raft_consensus.cc:2454] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d41eb1eaf78546208ffe61ff6827a825 in term 0.
I20260812 07:14:34.840701 26662 leader_election.cc:302] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a32fdd9471b4bfd937b5a7f2f861dea, d41eb1eaf78546208ffe61ff6827a825; no voters:
I20260812 07:14:34.841043 26897 raft_consensus.cc:2781] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 07:14:34.841172 26897 raft_consensus.cc:482] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 07:14:34.841264 26897 raft_consensus.cc:3037] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:34.843365 26897 raft_consensus.cc:504] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.843907 26897 leader_election.cc:288] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [CANDIDATE]: Term 1 election: Requested vote from peers b04243a014eb4b92b615a29bc6366f69 (127.21.181.65:36637), 1a32fdd9471b4bfd937b5a7f2f861dea (127.21.181.66:42259)
I20260812 07:14:34.844273 26318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec63e2269184e4cb6474b704bd3c535" candidate_uuid: "d41eb1eaf78546208ffe61ff6827a825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b04243a014eb4b92b615a29bc6366f69"
I20260812 07:14:34.844363 26549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ec63e2269184e4cb6474b704bd3c535" candidate_uuid: "d41eb1eaf78546208ffe61ff6827a825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea"
I20260812 07:14:34.844431 26318 raft_consensus.cc:3037] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:34.844556 26549 raft_consensus.cc:3037] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:34.846519 26318 raft_consensus.cc:2454] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41eb1eaf78546208ffe61ff6827a825 in term 1.
I20260812 07:14:34.846608 26549 raft_consensus.cc:2454] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d41eb1eaf78546208ffe61ff6827a825 in term 1.
I20260812 07:14:34.846894 26656 leader_election.cc:302] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b04243a014eb4b92b615a29bc6366f69, d41eb1eaf78546208ffe61ff6827a825; no voters:
I20260812 07:14:34.847203 26897 raft_consensus.cc:2781] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:34.847586 26897 raft_consensus.cc:686] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 1 LEADER]: Becoming Leader. State: Replica: d41eb1eaf78546208ffe61ff6827a825, State: Running, Role: LEADER
I20260812 07:14:34.848238 26897 consensus_queue.cc:237] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } }
I20260812 07:14:34.852802 26083 catalog_manager.cc:5495] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 reported cstate change: term changed from 0 to 1, leader changed from <none> to d41eb1eaf78546208ffe61ff6827a825 (127.21.181.67). New cstate: current_term: 1 leader_uuid: "d41eb1eaf78546208ffe61ff6827a825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d41eb1eaf78546208ffe61ff6827a825" member_type: VOTER last_known_addr { host: "127.21.181.67" port: 33709 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 } health_report { overall_health: UNKNOWN } } }
I20260812 07:14:34.878862 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.873705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.21.181.67:33709)
0812 07:14:34.873869 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.873874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.873894 (+ 20us) server_negotiation.cc:407] Connection header received
0812 07:14:34.874077 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.874081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.874168 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.874289 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:14:34.874633 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.875499 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.876850 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.877291 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.877414 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.877825 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:14:34.878354 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:14:34.878467 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.878547 (+ 80us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.878633 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260812 07:14:34.887387 26318 raft_consensus.cc:1261] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 1 FOLLOWER]: Refusing update from remote peer d41eb1eaf78546208ffe61ff6827a825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:34.887387 26549 raft_consensus.cc:1261] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 1 FOLLOWER]: Refusing update from remote peer d41eb1eaf78546208ffe61ff6827a825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:14:34.888495 26899 consensus_queue.cc:1035] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a32fdd9471b4bfd937b5a7f2f861dea" member_type: VOTER last_known_addr { host: "127.21.181.66" port: 42259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:34.888821 26897 consensus_queue.cc:1035] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b04243a014eb4b92b615a29bc6366f69" member_type: VOTER last_known_addr { host: "127.21.181.65" port: 36637 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:14:34.898640 26903 mvcc.cc:205] Tried to move back new op lower bound from 7317581311527981056 to 7317581311382130688. Current Snapshot: MvccSnapshot[applied={T|T < 7317581311527981056}]
I20260812 07:14:34.902283 26904 mvcc.cc:205] Tried to move back new op lower bound from 7317581311527981056 to 7317581311382130688. Current Snapshot: MvccSnapshot[applied={T|T < 7317581311527981056}]
I20260812 07:14:34.921396 26081 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:34.921712 26081 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260812 07:14:34.926920 26081 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5004951800ce4c3e8309e4514eaa52b2: Sending DeleteTablet for 3 replicas of tablet 7ec63e2269184e4cb6474b704bd3c535
I20260812 07:14:34.928037 26742 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec63e2269184e4cb6474b704bd3c535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:34 UTC) from {username='slave'} at 127.0.0.1:51730
I20260812 07:14:34.928040 26283 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec63e2269184e4cb6474b704bd3c535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:34 UTC) from {username='slave'} at 127.0.0.1:37938
I20260812 07:14:34.929436 26518 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ec63e2269184e4cb6474b704bd3c535 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-12 07:14:34 UTC) from {username='slave'} at 127.0.0.1:59834
I20260812 07:14:34.937184 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26180
I20260812 07:14:34.938064 26923 tablet_replica.cc:330] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: stopping tablet replica
I20260812 07:14:34.938445 26923 raft_consensus.cc:2229] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:34.939901 26921 tablet_replica.cc:330] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: stopping tablet replica
I20260812 07:14:34.940436 26923 raft_consensus.cc:2258] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:34.941587 26921 raft_consensus.cc:2229] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 07:14:34.942085 26921 raft_consensus.cc:2258] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:34.945308 26922 tablet_replica.cc:330] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: stopping tablet replica
I20260812 07:14:34.945724 26922 raft_consensus.cc:2229] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 1 LEADER]: Raft consensus shutting down.
I20260812 07:14:34.946444 26922 raft_consensus.cc:2258] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 07:14:34.946638 26921 ts_tablet_manager.cc:1905] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:34.946866 26923 ts_tablet_manager.cc:1905] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:34.950323 26922 ts_tablet_manager.cc:1905] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260812 07:14:34.957240 26922 ts_tablet_manager.cc:1918] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:34.957427 26922 log.cc:1196] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-2/wal/wals/7ec63e2269184e4cb6474b704bd3c535
I20260812 07:14:34.957959 26922 ts_tablet_manager.cc:1939] T 7ec63e2269184e4cb6474b704bd3c535 P d41eb1eaf78546208ffe61ff6827a825: Deleting consensus metadata
I20260812 07:14:34.959618 26064 catalog_manager.cc:4841] TS d41eb1eaf78546208ffe61ff6827a825 (127.21.181.67:33709): tablet 7ec63e2269184e4cb6474b704bd3c535 (table test-table [id=9d2b58cb56ef4d2581bcc017b67a29a6]) successfully deleted
I20260812 07:14:34.961208 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:34.938665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37950 (local address 127.21.181.65:36637)
0812 07:14:34.938851 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:34.938858 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:34.940612 (+ 1754us) server_negotiation.cc:407] Connection header received
0812 07:14:34.940890 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:34.940896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:34.941015 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:34.941090 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:34.941786 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.943021 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.946681 (+ 3660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:34.947257 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:34.947352 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:34.959563 (+ 12211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:34.959574 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:34.959589 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:34.960316 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:34.960371 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:34.960395 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:34.960487 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:34.960724 (+ 237us) server_negotiation.cc:299] Negotiation successful
0812 07:14:34.960836 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:14:34.956015 26923 ts_tablet_manager.cc:1918] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:34.961796 26923 log.cc:1196] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-1/wal/wals/7ec63e2269184e4cb6474b704bd3c535
I20260812 07:14:34.962218 26394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:34.962327 26923 ts_tablet_manager.cc:1939] T 7ec63e2269184e4cb6474b704bd3c535 P 1a32fdd9471b4bfd937b5a7f2f861dea: Deleting consensus metadata
I20260812 07:14:34.963652 26066 catalog_manager.cc:4841] TS 1a32fdd9471b4bfd937b5a7f2f861dea (127.21.181.66:42259): tablet 7ec63e2269184e4cb6474b704bd3c535 (table test-table [id=9d2b58cb56ef4d2581bcc017b67a29a6]) successfully deleted
I20260812 07:14:35.049458 26921 ts_tablet_manager.cc:1918] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260812 07:14:35.049727 26921 log.cc:1196] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Deleting WAL directory at /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786518562800324-22229-0/minicluster-data/ts-0/wal/wals/7ec63e2269184e4cb6474b704bd3c535
I20260812 07:14:35.050468 26921 ts_tablet_manager.cc:1939] T 7ec63e2269184e4cb6474b704bd3c535 P b04243a014eb4b92b615a29bc6366f69: Deleting consensus metadata
I20260812 07:14:35.050526 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26180
W20260812 07:14:35.082731 26061 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.21.181.65:36637 (error 108)
I20260812 07:14:35.082998 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26412
I20260812 07:14:35.085168 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.084445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:37962)
0812 07:14:35.084872 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.084990 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":222,"threads_started":1}
W20260812 07:14:35.085393 26061 catalog_manager.cc:4584] TS b04243a014eb4b92b615a29bc6366f69 (127.21.181.65:36637): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ec63e2269184e4cb6474b704bd3c535: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
I20260812 07:14:35.090914 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.084166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59838 (local address 127.21.181.66:42259)
0812 07:14:35.084366 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:35.084372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:35.084396 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:14:35.084502 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:35.084508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:35.084615 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:35.084717 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:35.085463 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:35.086281 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:35.088047 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:35.088489 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:35.088585 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:35.089615 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:35.089623 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:35.089638 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:35.090234 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:35.090283 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:35.090305 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:35.090389 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:35.090532 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 07:14:35.090621 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260812 07:14:35.091579 26614 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:35.139138 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.138619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:37972)
0812 07:14:35.138854 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.138972 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:14:35.140354 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.139918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:37986)
0812 07:14:35.140110 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.140196 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260812 07:14:35.178224 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26412
I20260812 07:14:35.200892 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.200355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:37998)
0812 07:14:35.200584 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.200699 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260812 07:14:35.202106 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.201679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:38012)
0812 07:14:35.201832 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.201925 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260812 07:14:35.208719 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26623
I20260812 07:14:35.216344 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.209529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51746 (local address 127.21.181.67:33709)
0812 07:14:35.209717 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:35.209723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:35.209749 (+ 26us) server_negotiation.cc:407] Connection header received
0812 07:14:35.209816 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:35.209821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:35.209934 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:35.210047 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:35.210430 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:35.211609 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:35.212893 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:35.213548 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:35.213672 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:35.214643 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:35.214656 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:35.214676 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:35.215532 (+ 856us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:35.215598 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:35.215628 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:35.215735 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:35.215812 (+ 77us) server_negotiation.cc:299] Negotiation successful
0812 07:14:35.215914 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260812 07:14:35.217093 26842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:35.282776 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.282181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:38028)
0812 07:14:35.282453 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.282577 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:14:35.284102 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.283588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:38030)
0812 07:14:35.283798 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.283904 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260812 07:14:35.353461 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26623
I20260812 07:14:35.379009 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26040
I20260812 07:14:35.380213 26137 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:35.485693 26928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:35.484867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.65:36637 (local address 127.0.0.1:38038)
0812 07:14:35.485339 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0812 07:14:35.485464 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.65:36637: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216}
I20260812 07:14:35.486091 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26040
2026-08-12T07:14:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3043 ms)
[----------] 6 tests from EncryptionPolicyTest (21985 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:35 dist-test-slave-lgr5 krb5kdc[26941](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:35 dist-test-slave-lgr5 krb5kdc[26941](info): set up 2 sockets
Aug 12 07:14:35 dist-test-slave-lgr5 krb5kdc[26941](info): commencing operation
krb5kdc: starting...
W20260812 07:14:36.757201 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:36 dist-test-slave-lgr5 krb5kdc[26941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:37.037343 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34781
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34781
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:37.366299 26964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:37.366581 26964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:37.366722 26964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:37.373807 26964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:37.373910 26964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:37.373986 26964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:37.374070 26964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:37.374125 26964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:37.383646 26964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34781
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34781
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:37.384869 26964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:37.386142 26964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:37.397370 26981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:37.397472 26975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:37.397432 26983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:37.398039 26964 server_base.cc:1034] running on GCE node
I20260812 07:14:37.398702 26964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:37.399959 26964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:37.401151 26964 hybrid_clock.cc:648] HybridClock initialized: now 1786518877401112 us; error 48 us; skew 500 ppm
I20260812 07:14:37.401473 26964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:37 dist-test-slave-lgr5 krb5kdc[26941](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518877, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:37.407361 26964 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:37.408762 26964 webserver.cc:460] Webserver started at http://127.0.0.1:45979/ using document root <none> and password file <none>
I20260812 07:14:37.409392 26964 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:37.409511 26964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:37.409801 26964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:37.412724 26964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "73f18fed2c84478ab10cdb104327f526"
format_stamp: "Formatted at 2026-08-12 07:14:37 on dist-test-slave-lgr5"
server_key: "73626f2b83feaa1924854668d37db3c4"
server_key_iv: "2e60432fe00d8e40e50169694a6c8e28"
server_key_version: "encryptionkey@0"
I20260812 07:14:37.413466 26964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "73f18fed2c84478ab10cdb104327f526"
format_stamp: "Formatted at 2026-08-12 07:14:37 on dist-test-slave-lgr5"
server_key: "73626f2b83feaa1924854668d37db3c4"
server_key_iv: "2e60432fe00d8e40e50169694a6c8e28"
server_key_version: "encryptionkey@0"
I20260812 07:14:37.418381 26964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260812 07:14:37.421617 26998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:37.422775 26964 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:14:37.422950 26964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "73f18fed2c84478ab10cdb104327f526"
format_stamp: "Formatted at 2026-08-12 07:14:37 on dist-test-slave-lgr5"
server_key: "73626f2b83feaa1924854668d37db3c4"
server_key_iv: "2e60432fe00d8e40e50169694a6c8e28"
server_key_version: "encryptionkey@0"
I20260812 07:14:37.423133 26964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:37.445216 26964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:37.448845 26964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:37.449088 26964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:37.465986 26964 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34781
I20260812 07:14:37.466022 27089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34781 every 8 connection(s)
I20260812 07:14:37.467228 26964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:37.470665 27090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:37.471709 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 26964
I20260812 07:14:37.471925 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:37.472602 22229 external_mini_cluster.cc:1428] Setting key 59484501a9d480330eaf6c42f95799ee
I20260812 07:14:37.483927 27090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526: Bootstrap starting.
Aug 12 07:14:37 dist-test-slave-lgr5 krb5kdc[26941](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:14:37.486752 27090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:37.487799 27090 log.cc:824] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:37.490517 27090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526: No bootstrap required, opened a new log
I20260812 07:14:37.494206 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:37.474489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44469 (local address 127.0.0.1:34781)
0812 07:14:37.475288 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:37.475301 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:37.475342 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:14:37.476928 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:37.476949 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:37.477325 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:37.477620 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:37.479341 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:37.480694 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:37.482472 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:37.483116 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:37.483236 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:37.487182 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:37.487189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:37.487202 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:37.487240 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:37.490209 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:37.490979 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:37.490985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:37.490991 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:37.491156 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:37.491621 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:37.491625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:37.491627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:37.492168 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:37.492497 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:37.492992 (+ 495us) server_negotiation.cc:299] Negotiation successful
0812 07:14:37.493274 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":362,"threads_started":1}
I20260812 07:14:37.497936 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:37.498493 27090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f18fed2c84478ab10cdb104327f526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34781 } }
I20260812 07:14:37.498805 27090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:37.498900 27090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73f18fed2c84478ab10cdb104327f526, State: Initialized, Role: FOLLOWER
I20260812 07:14:37.499413 27090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f18fed2c84478ab10cdb104327f526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34781 } }
I20260812 07:14:37.499619 27090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:37.499739 27090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:37.499886 27090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:37.502050 27090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f18fed2c84478ab10cdb104327f526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34781 } }
I20260812 07:14:37.502527 27090 leader_election.cc:302] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73f18fed2c84478ab10cdb104327f526; no voters:
I20260812 07:14:37.503238 27090 leader_election.cc:288] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:37.503291 27110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:37.505366 27090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:37.509334 27110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [term 1 LEADER]: Becoming Leader. State: Replica: 73f18fed2c84478ab10cdb104327f526, State: Running, Role: LEADER
I20260812 07:14:37.510046 27110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f18fed2c84478ab10cdb104327f526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34781 } }
I20260812 07:14:37.513056 27112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73f18fed2c84478ab10cdb104327f526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f18fed2c84478ab10cdb104327f526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34781 } } }
I20260812 07:14:37.513269 27113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73f18fed2c84478ab10cdb104327f526. Latest consensus state: current_term: 1 leader_uuid: "73f18fed2c84478ab10cdb104327f526" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73f18fed2c84478ab10cdb104327f526" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34781 } } }
I20260812 07:14:37.513756 27113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:37.513758 27112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:37.515509 27141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:37.520855 27141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:37.521929 27094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:37.502541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50827 (local address 127.0.0.1:34781)
0812 07:14:37.502756 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:37.502762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:37.503228 (+ 466us) server_negotiation.cc:407] Connection header received
0812 07:14:37.503427 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:37.503432 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:37.503524 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:37.503629 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:37.505603 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:37.506844 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:37.508726 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:37.509491 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:37.509632 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:37.512634 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:37.512641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:37.512645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:37.512679 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:37.517943 (+ 5264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:37.518894 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:37.518899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:37.518902 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:37.519064 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:37.519692 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:37.519697 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:37.519699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:37.520105 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:37.521179 (+ 1074us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:37.521385 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 07:14:37.521569 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:14:37.528442 27141 catalog_manager.cc:1347] Generated new cluster ID: 8fef44f2bcc741ba92c604b051661136
I20260812 07:14:37.528537 27141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:37.552606 27141 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:37.553769 27141 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:37.562410 27141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73f18fed2c84478ab10cdb104327f526: Generated new TSK 0
I20260812 07:14:37.563133 27141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:37.569433 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 26964
I20260812 07:14:37.570937 27081 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:37.659528 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 26964
W20260812 07:14:37.686949 26936 connection.cc:535] client connection to 127.0.0.1:34781 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34781): Connection reset by peer (error 104)
2026-08-12T07:14:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2176 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:37 dist-test-slave-lgr5 krb5kdc[27153](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:37 dist-test-slave-lgr5 krb5kdc[27153](info): set up 2 sockets
Aug 12 07:14:37 dist-test-slave-lgr5 krb5kdc[27153](info): commencing operation
krb5kdc: starting...
W20260812 07:14:38.919442 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:39 dist-test-slave-lgr5 krb5kdc[27153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:39.189424 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:39.510394 27177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:39.510684 27177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:39.510826 27177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:39.517858 27177 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:39.517982 27177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:39.518062 27177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:39.518146 27177 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:39.518208 27177 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:39.527522 27177 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:39.528738 27177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:39.529981 27177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:39.541086 27187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:39.541148 27182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:39.541167 27184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:39.542156 27177 server_base.cc:1034] running on GCE node
I20260812 07:14:39.542843 27177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:39.544140 27177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:39.545341 27177 hybrid_clock.cc:648] HybridClock initialized: now 1786518879545292 us; error 42 us; skew 500 ppm
I20260812 07:14:39.545670 27177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:39 dist-test-slave-lgr5 krb5kdc[27153](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518879, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:39.548655 27177 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:39.549999 27177 webserver.cc:460] Webserver started at http://127.0.0.1:46863/ using document root <none> and password file <none>
I20260812 07:14:39.550613 27177 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:39.550746 27177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:39.551010 27177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:39.553802 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "12d85e290b264ed19960240581a9e0b6"
format_stamp: "Formatted at 2026-08-12 07:14:39 on dist-test-slave-lgr5"
server_key: "439b85b603618a593d5ca864e20d4d13"
server_key_iv: "ab6e567eb67bdcd6fc480e85e56aefc7"
server_key_version: "encryptionkey@0"
I20260812 07:14:39.554509 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "12d85e290b264ed19960240581a9e0b6"
format_stamp: "Formatted at 2026-08-12 07:14:39 on dist-test-slave-lgr5"
server_key: "439b85b603618a593d5ca864e20d4d13"
server_key_iv: "ab6e567eb67bdcd6fc480e85e56aefc7"
server_key_version: "encryptionkey@0"
I20260812 07:14:39.559271 27177 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260812 07:14:39.562510 27201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:39.563692 27177 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:14:39.563849 27177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "12d85e290b264ed19960240581a9e0b6"
format_stamp: "Formatted at 2026-08-12 07:14:39 on dist-test-slave-lgr5"
server_key: "439b85b603618a593d5ca864e20d4d13"
server_key_iv: "ab6e567eb67bdcd6fc480e85e56aefc7"
server_key_version: "encryptionkey@0"
I20260812 07:14:39.564024 27177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:39.576689 27177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:39.580621 27177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:39.580858 27177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:39.598016 27177 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33107
I20260812 07:14:39.598044 27292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33107 every 8 connection(s)
I20260812 07:14:39.599272 27177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:39.601635 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27177
I20260812 07:14:39.601805 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:39.602219 22229 external_mini_cluster.cc:1428] Setting key 69b1af9c294ba0731776824ec8276739
I20260812 07:14:39.602902 27294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:39.614589 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:39.603828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59709 (local address 127.0.0.1:33107)
0812 07:14:39.604572 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:39.604583 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:39.604620 (+ 37us) server_negotiation.cc:407] Connection header received
0812 07:14:39.606106 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:39.606127 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:39.606486 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:39.606805 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:39.607331 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:39.608720 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:39.610475 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:39.611103 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:39.611229 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:39.612465 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:39.612472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:39.612484 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:39.613015 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:39.613089 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:39.613114 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:39.613231 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:39.613509 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:14:39.613694 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":346,"threads_started":1}
I20260812 07:14:39.615770 27294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6: Bootstrap starting.
I20260812 07:14:39.618052 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:39.618927 27294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:39.620069 27294 log.cc:824] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:39.622680 27294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6: No bootstrap required, opened a new log
I20260812 07:14:39.629197 27300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:39.622870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47257 (local address 127.0.0.1:33107)
0812 07:14:39.623045 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:39.623050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:39.623501 (+ 451us) server_negotiation.cc:407] Connection header received
0812 07:14:39.623604 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:39.623609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:39.623674 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:39.623783 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:39.624284 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:39.625417 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:39.626688 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:39.627376 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:39.627516 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:39.628469 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:39.628487 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:39.628492 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:39.628513 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:39.628573 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:39.628593 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:39.628708 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:39.628827 (+ 119us) server_negotiation.cc:299] Negotiation successful
0812 07:14:39.628879 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:14:39.631640 27294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d85e290b264ed19960240581a9e0b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260812 07:14:39.631958 27294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:39.632047 27294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d85e290b264ed19960240581a9e0b6, State: Initialized, Role: FOLLOWER
I20260812 07:14:39.632561 27294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d85e290b264ed19960240581a9e0b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260812 07:14:39.632755 27294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:39.632859 27294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:39.632992 27294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:39.634964 27294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d85e290b264ed19960240581a9e0b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260812 07:14:39.635444 27294 leader_election.cc:302] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d85e290b264ed19960240581a9e0b6; no voters:
I20260812 07:14:39.635957 27294 leader_election.cc:288] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:39.636133 27313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:39.637145 27313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [term 1 LEADER]: Becoming Leader. State: Replica: 12d85e290b264ed19960240581a9e0b6, State: Running, Role: LEADER
I20260812 07:14:39.637621 27313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d85e290b264ed19960240581a9e0b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } }
I20260812 07:14:39.637948 27294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:39.640060 27317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d85e290b264ed19960240581a9e0b6. Latest consensus state: current_term: 1 leader_uuid: "12d85e290b264ed19960240581a9e0b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d85e290b264ed19960240581a9e0b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } } }
I20260812 07:14:39.640498 27317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:39.640942 27314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d85e290b264ed19960240581a9e0b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d85e290b264ed19960240581a9e0b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33107 } } }
I20260812 07:14:39.641196 27314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:39.652823 27330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:39.657418 27330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:39.665918 27330 catalog_manager.cc:1347] Generated new cluster ID: abf5f9a291504186bfd69028c0d917d9
I20260812 07:14:39.666018 27330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:39.683409 27330 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:39.684649 27330 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:39.691951 27330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d85e290b264ed19960240581a9e0b6: Generated new TSK 0
I20260812 07:14:39.692706 27330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:39.702263 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27177
W20260812 07:14:39.703644 27286 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59709
W20260812 07:14:39.704334 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27177: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:39.704499 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27177
W20260812 07:14:39.730232 27150 connection.cc:535] client connection to 127.0.0.1:33107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33107): Connection reset by peer (error 104)
2026-08-12T07:14:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2039 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:39 dist-test-slave-lgr5 krb5kdc[27350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:39 dist-test-slave-lgr5 krb5kdc[27350](info): set up 2 sockets
Aug 12 07:14:39 dist-test-slave-lgr5 krb5kdc[27350](info): commencing operation
krb5kdc: starting...
W20260812 07:14:40.985178 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:41 dist-test-slave-lgr5 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:41.260170 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:41.583168 27373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:41.583444 27373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:41.583575 27373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:41.590617 27373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:41.590723 27373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:41.590802 27373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:41.590885 27373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:41.590942 27373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:41.600507 27373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44835
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:41.601732 27373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:41.602983 27373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:41.613936 27384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:41.613873 27388 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:41.613902 27386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:41.614998 27373 server_base.cc:1034] running on GCE node
I20260812 07:14:41.615733 27373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:41.617064 27373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:41.618268 27373 hybrid_clock.cc:648] HybridClock initialized: now 1786518881618214 us; error 39 us; skew 500 ppm
I20260812 07:14:41.618618 27373 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:41 dist-test-slave-lgr5 krb5kdc[27350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518881, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:41.621625 27373 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:41.622974 27373 webserver.cc:460] Webserver started at http://127.0.0.1:39535/ using document root <none> and password file <none>
I20260812 07:14:41.623631 27373 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:41.623747 27373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:41.624044 27373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:41.626911 27373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "c0fa1179726d4a17b1d16f54ac039ca0"
format_stamp: "Formatted at 2026-08-12 07:14:41 on dist-test-slave-lgr5"
server_key: "e0e14b9230104dfec2fe442b9c6fea40"
server_key_iv: "dd99ca10ea3e5eee9e1ca9da150f1a7d"
server_key_version: "encryptionkey@0"
I20260812 07:14:41.627652 27373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "c0fa1179726d4a17b1d16f54ac039ca0"
format_stamp: "Formatted at 2026-08-12 07:14:41 on dist-test-slave-lgr5"
server_key: "e0e14b9230104dfec2fe442b9c6fea40"
server_key_iv: "dd99ca10ea3e5eee9e1ca9da150f1a7d"
server_key_version: "encryptionkey@0"
I20260812 07:14:41.632473 27373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.002s
I20260812 07:14:41.635627 27400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:41.636852 27373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:14:41.637010 27373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "c0fa1179726d4a17b1d16f54ac039ca0"
format_stamp: "Formatted at 2026-08-12 07:14:41 on dist-test-slave-lgr5"
server_key: "e0e14b9230104dfec2fe442b9c6fea40"
server_key_iv: "dd99ca10ea3e5eee9e1ca9da150f1a7d"
server_key_version: "encryptionkey@0"
I20260812 07:14:41.637180 27373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:41.642683 27373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:41.646533 27373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:41.646793 27373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:41.663683 27373 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44835
I20260812 07:14:41.663723 27501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44835 every 8 connection(s)
I20260812 07:14:41.664990 27373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:41.668457 27502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:41.674216 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27373
I20260812 07:14:41.674400 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:41.674809 22229 external_mini_cluster.cc:1428] Setting key cacb61b81a3a67d4e8d46e01b645c06a
I20260812 07:14:41.678576 27502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0: Bootstrap starting.
I20260812 07:14:41.681882 27502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:41.681942 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:41.676503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60609 (local address 127.0.0.1:44835)
0812 07:14:41.677274 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:41.677285 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:41.677326 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:14:41.678928 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:41.678950 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:41.679242 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:41.679539 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:41.680566 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:41.680572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:41.680585 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:41.680598 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:41.680652 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:41.680674 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:41.680844 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:41.680937 (+ 93us) server_negotiation.cc:299] Negotiation successful
0812 07:14:41.681089 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":349,"threads_started":1}
I20260812 07:14:41.683156 27502 log.cc:824] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:41.685616 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:41.686017 27502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0: No bootstrap required, opened a new log
I20260812 07:14:41.691803 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:41.689634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54951 (local address 127.0.0.1:44835)
0812 07:14:41.689814 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:41.689820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:41.690136 (+ 316us) server_negotiation.cc:407] Connection header received
0812 07:14:41.690278 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:41.690281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:41.690324 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:41.690404 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:41.691232 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:41.691236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:41.691239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:41.691252 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:41.691289 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:41.691307 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:41.691392 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:41.691503 (+ 111us) server_negotiation.cc:299] Negotiation successful
0812 07:14:41.691548 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260812 07:14:41.694129 27502 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260812 07:14:41.694458 27502 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:41.694552 27502 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0fa1179726d4a17b1d16f54ac039ca0, State: Initialized, Role: FOLLOWER
I20260812 07:14:41.695091 27502 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260812 07:14:41.695309 27502 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:41.695434 27502 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:41.695586 27502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:41.697762 27502 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260812 07:14:41.698235 27502 leader_election.cc:302] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0fa1179726d4a17b1d16f54ac039ca0; no voters:
I20260812 07:14:41.698727 27502 leader_election.cc:288] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:41.698839 27518 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:41.699437 27518 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [term 1 LEADER]: Becoming Leader. State: Replica: c0fa1179726d4a17b1d16f54ac039ca0, State: Running, Role: LEADER
I20260812 07:14:41.699928 27518 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } }
I20260812 07:14:41.700623 27502 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:41.702661 27522 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0fa1179726d4a17b1d16f54ac039ca0. Latest consensus state: current_term: 1 leader_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260812 07:14:41.703234 27522 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:41.702857 27521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0fa1179726d4a17b1d16f54ac039ca0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44835 } } }
I20260812 07:14:41.704548 27521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:41.709247 27531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:41.714223 27531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:41.722383 27531 catalog_manager.cc:1347] Generated new cluster ID: 190c9f8b8bb74cdd837701fc6f293140
I20260812 07:14:41.722479 27531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:41.745170 27531 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:41.745558 27531 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:41.760149 27531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0fa1179726d4a17b1d16f54ac039ca0: Generated new TSK 0
I20260812 07:14:41.760888 27531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:41.775430 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27373
W20260812 07:14:41.776880 27496 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60609
W20260812 07:14:41.777978 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27373: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:41.778117 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27373
2026-08-12T07:14:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2078 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:41 dist-test-slave-lgr5 krb5kdc[27556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:41 dist-test-slave-lgr5 krb5kdc[27556](info): set up 2 sockets
Aug 12 07:14:41 dist-test-slave-lgr5 krb5kdc[27556](info): commencing operation
krb5kdc: starting...
W20260812 07:14:43.041661 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:43 dist-test-slave-lgr5 krb5kdc[27556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:43.303227 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:43.631177 27584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:43.631464 27584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:43.631608 27584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:43.638732 27584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:43.638839 27584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:43.638921 27584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:43.639014 27584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:43.639076 27584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:43.648543 27584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:43.649627 27584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:43.650847 27584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:43.662130 27591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:43.662211 27592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:43.662177 27595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:43.662899 27584 server_base.cc:1034] running on GCE node
I20260812 07:14:43.663604 27584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:43.664927 27584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:43.666139 27584 hybrid_clock.cc:648] HybridClock initialized: now 1786518883666096 us; error 36 us; skew 500 ppm
I20260812 07:14:43.666494 27584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:43 dist-test-slave-lgr5 krb5kdc[27556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518883, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:43.673406 27584 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:43.674880 27584 webserver.cc:460] Webserver started at http://127.0.0.1:34177/ using document root <none> and password file <none>
I20260812 07:14:43.675534 27584 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:43.675664 27584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:43.675958 27584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:43.678933 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "1ff38eb8246e4d2e96f52bf263b5631b"
format_stamp: "Formatted at 2026-08-12 07:14:43 on dist-test-slave-lgr5"
server_key: "81d16043d42a939a5fc8ed4a910e5c9e"
server_key_iv: "49738ff614325d013038cfd3bb6398ec"
server_key_version: "encryptionkey@0"
I20260812 07:14:43.679726 27584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "1ff38eb8246e4d2e96f52bf263b5631b"
format_stamp: "Formatted at 2026-08-12 07:14:43 on dist-test-slave-lgr5"
server_key: "81d16043d42a939a5fc8ed4a910e5c9e"
server_key_iv: "49738ff614325d013038cfd3bb6398ec"
server_key_version: "encryptionkey@0"
I20260812 07:14:43.684799 27584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260812 07:14:43.688149 27605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:43.689402 27584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:14:43.689579 27584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "1ff38eb8246e4d2e96f52bf263b5631b"
format_stamp: "Formatted at 2026-08-12 07:14:43 on dist-test-slave-lgr5"
server_key: "81d16043d42a939a5fc8ed4a910e5c9e"
server_key_iv: "49738ff614325d013038cfd3bb6398ec"
server_key_version: "encryptionkey@0"
I20260812 07:14:43.689764 27584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:43.711287 27584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:43.715070 27584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:43.715356 27584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:43.733371 27584 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44601
I20260812 07:14:43.733388 27698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44601 every 8 connection(s)
I20260812 07:14:43.734668 27584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:43.735183 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27584
I20260812 07:14:43.735397 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:43.735983 22229 external_mini_cluster.cc:1428] Setting key abfb4a69fe00b9b075e2c760bb2476b4
I20260812 07:14:43.738754 27699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:14:43 dist-test-slave-lgr5 krb5kdc[27556](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:14:43.751081 27699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b: Bootstrap starting.
I20260812 07:14:43.754948 27699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:43.756203 27699 log.cc:824] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:43.759341 27699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b: No bootstrap required, opened a new log
I20260812 07:14:43.759594 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:43.737621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58015 (local address 127.0.0.1:44601)
0812 07:14:43.738411 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:43.738422 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:43.738463 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:14:43.740225 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:43.740248 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:43.740609 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:43.740936 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:43.742732 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:43.744210 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:43.745953 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:43.746617 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:43.746758 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:43.751512 (+ 4754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:43.751526 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:43.751544 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:43.751587 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:43.755308 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:43.756020 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:43.756028 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:43.756036 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:43.756261 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:43.756709 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:43.756715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:43.756718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:43.757271 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:43.757514 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:43.757976 (+ 462us) server_negotiation.cc:299] Negotiation successful
0812 07:14:43.758372 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":370,"threads_started":1}
I20260812 07:14:43.763273 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:43.766680 27699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260812 07:14:43.766974 27699 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:43.767076 27699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ff38eb8246e4d2e96f52bf263b5631b, State: Initialized, Role: FOLLOWER
I20260812 07:14:43.767616 27699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260812 07:14:43.767910 27699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:43.768047 27699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:43.768247 27699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:43.770548 27699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260812 07:14:43.771086 27699 leader_election.cc:302] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ff38eb8246e4d2e96f52bf263b5631b; no voters:
I20260812 07:14:43.771749 27699 leader_election.cc:288] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:43.771831 27714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:43.773823 27699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:43.775714 27714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [term 1 LEADER]: Becoming Leader. State: Replica: 1ff38eb8246e4d2e96f52bf263b5631b, State: Running, Role: LEADER
I20260812 07:14:43.776376 27714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } }
I20260812 07:14:43.779323 27716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ff38eb8246e4d2e96f52bf263b5631b. Latest consensus state: current_term: 1 leader_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } } }
I20260812 07:14:43.779727 27716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:43.780251 27715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ff38eb8246e4d2e96f52bf263b5631b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44601 } } }
I20260812 07:14:43.780516 27715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:43.784559 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:43.767663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50321 (local address 127.0.0.1:44601)
0812 07:14:43.767834 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:43.767839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:43.768305 (+ 466us) server_negotiation.cc:407] Connection header received
0812 07:14:43.768465 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:43.768470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:43.768558 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:43.768658 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:43.770073 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:43.771115 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:43.772664 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:43.773224 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:43.773310 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:43.776198 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:43.776206 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:43.776212 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:43.776265 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:43.780714 (+ 4449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:43.781654 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:43.781662 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:43.781666 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:43.781904 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:43.782429 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:43.782435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:43.782439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:43.782889 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:43.783187 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:43.783863 (+ 676us) server_negotiation.cc:299] Negotiation successful
0812 07:14:43.784122 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260812 07:14:43.789034 27735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:43.793310 27735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:43.802048 27735 catalog_manager.cc:1347] Generated new cluster ID: 01c6ea80dad446e79264511fc5244ddd
I20260812 07:14:43.802160 27735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:43.823266 27735 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:43.825071 27735 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:43.832883 27735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ff38eb8246e4d2e96f52bf263b5631b: Generated new TSK 0
I20260812 07:14:43.833799 27735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:43.843657 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27584
I20260812 07:14:43.844663 27690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:43.932895 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27584
2026-08-12T07:14:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2150 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:44 dist-test-slave-lgr5 krb5kdc[27753](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:44 dist-test-slave-lgr5 krb5kdc[27753](info): set up 2 sockets
Aug 12 07:14:44 dist-test-slave-lgr5 krb5kdc[27753](info): commencing operation
krb5kdc: starting...
W20260812 07:14:45.226729 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:45 dist-test-slave-lgr5 krb5kdc[27753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:45.490940 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:45.819805 27775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:45.820133 27775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:45.820256 27775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:45.827482 27775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:45.827596 27775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:45.827657 27775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:45.827732 27775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:45.827782 27775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:45.837666 27775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:45.838918 27775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:45.840224 27775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:45.851418 27791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:45.851462 27795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:45.851483 27788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:45.852321 27775 server_base.cc:1034] running on GCE node
I20260812 07:14:45.853004 27775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:45.854238 27775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:45.855434 27775 hybrid_clock.cc:648] HybridClock initialized: now 1786518885855399 us; error 27 us; skew 500 ppm
I20260812 07:14:45.855755 27775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:45 dist-test-slave-lgr5 krb5kdc[27753](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:45.858698 27775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:45.860139 27775 webserver.cc:460] Webserver started at http://127.0.0.1:36581/ using document root <none> and password file <none>
I20260812 07:14:45.860764 27775 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:45.860880 27775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:45.861169 27775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:45.863965 27775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383"
format_stamp: "Formatted at 2026-08-12 07:14:45 on dist-test-slave-lgr5"
server_key: "403c9f57a4957c8162541074073a0ca9"
server_key_iv: "57e2f23fdef8b29c1c66e48faff6ad4a"
server_key_version: "encryptionkey@0"
I20260812 07:14:45.864755 27775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383"
format_stamp: "Formatted at 2026-08-12 07:14:45 on dist-test-slave-lgr5"
server_key: "403c9f57a4957c8162541074073a0ca9"
server_key_iv: "57e2f23fdef8b29c1c66e48faff6ad4a"
server_key_version: "encryptionkey@0"
I20260812 07:14:45.869513 27775 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260812 07:14:45.872702 27814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:45.873904 27775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:14:45.874064 27775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383"
format_stamp: "Formatted at 2026-08-12 07:14:45 on dist-test-slave-lgr5"
server_key: "403c9f57a4957c8162541074073a0ca9"
server_key_iv: "57e2f23fdef8b29c1c66e48faff6ad4a"
server_key_version: "encryptionkey@0"
I20260812 07:14:45.874228 27775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:45.901247 27775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:45.904985 27775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:45.905241 27775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:45.922186 27775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38311
I20260812 07:14:45.922216 27901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38311 every 8 connection(s)
I20260812 07:14:45.923461 27775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:45.926964 27903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:45.933122 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27775
I20260812 07:14:45.933321 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:45.933892 22229 external_mini_cluster.cc:1428] Setting key 6a16b57d8ebf56ab487e3a5e2d102683
I20260812 07:14:45.937780 27903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383: Bootstrap starting.
I20260812 07:14:45.940762 27903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:45.941692 27903 log.cc:824] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:45.944370 27903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383: No bootstrap required, opened a new log
I20260812 07:14:45.946234 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:45.935767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54563 (local address 127.0.0.1:38311)
0812 07:14:45.936521 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:45.936532 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:45.936572 (+ 40us) server_negotiation.cc:407] Connection header received
0812 07:14:45.938231 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:45.938253 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:45.938614 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:45.938943 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:45.939474 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:45.940781 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:45.942127 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:45.942808 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:45.942966 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:45.944114 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:45.944126 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:45.944142 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:45.944677 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:45.944753 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:45.944780 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:45.944887 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:45.945116 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 07:14:45.945336 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":354,"threads_started":1}
I20260812 07:14:45.949667 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:45.951982 27903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260812 07:14:45.952412 27903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:45.952529 27903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77a237fb5ad84d7cb9ca2fc2fb7d0383, State: Initialized, Role: FOLLOWER
I20260812 07:14:45.953101 27903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260812 07:14:45.953307 27903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:45.953467 27903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:45.953660 27903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:45.956569 27903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260812 07:14:45.956998 27903 leader_election.cc:302] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77a237fb5ad84d7cb9ca2fc2fb7d0383; no voters:
I20260812 07:14:45.957609 27903 leader_election.cc:288] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:45.957696 27926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:45.958844 27926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [term 1 LEADER]: Becoming Leader. State: Replica: 77a237fb5ad84d7cb9ca2fc2fb7d0383, State: Running, Role: LEADER
I20260812 07:14:45.959415 27926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } }
I20260812 07:14:45.959728 27903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:45.960541 27908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:45.953824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55373 (local address 127.0.0.1:38311)
0812 07:14:45.953987 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:45.953992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:45.954372 (+ 380us) server_negotiation.cc:407] Connection header received
0812 07:14:45.954490 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:45.954495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:45.954572 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:45.954671 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:45.955086 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:45.956146 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:45.957511 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:45.958281 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:45.958394 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:45.959696 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:45.959702 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:45.959707 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:45.959731 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:45.959773 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:45.959795 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:45.959882 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:45.960111 (+ 229us) server_negotiation.cc:299] Negotiation successful
0812 07:14:45.960197 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260812 07:14:45.966189 27927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } } }
I20260812 07:14:45.966567 27927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:45.966876 27928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77a237fb5ad84d7cb9ca2fc2fb7d0383. Latest consensus state: current_term: 1 leader_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77a237fb5ad84d7cb9ca2fc2fb7d0383" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38311 } } }
I20260812 07:14:45.967133 27928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383 [sys.catalog]: This master's current role is: LEADER
W20260812 07:14:45.970397 27948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:14:45.970530 27948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:14:45.970649 27949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:45.972363 27949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:45.981470 27949 catalog_manager.cc:1347] Generated new cluster ID: 63fe6d780963429fb859fb658e2eaf64
I20260812 07:14:45.981573 27949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:45.999495 27949 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:46.000797 27949 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:46.025836 27949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77a237fb5ad84d7cb9ca2fc2fb7d0383: Generated new TSK 0
I20260812 07:14:46.026728 27949 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:46.042423 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27775
W20260812 07:14:46.043499 27895 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54563
W20260812 07:14:46.044155 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27775: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:46.044303 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27775
2026-08-12T07:14:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2107 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:46 dist-test-slave-lgr5 krb5kdc[27957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:46 dist-test-slave-lgr5 krb5kdc[27957](info): set up 2 sockets
Aug 12 07:14:46 dist-test-slave-lgr5 krb5kdc[27957](info): commencing operation
krb5kdc: starting...
W20260812 07:14:47.295190 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:47 dist-test-slave-lgr5 krb5kdc[27957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:47.556497 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:47.873003 27982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:47.873274 27982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:47.873416 27982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:47.880394 27982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:47.880510 27982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:47.880586 27982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:47.880668 27982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:47.880728 27982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:47.890106 27982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:47.891266 27982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:47.892540 27982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:47.903344 27998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:47.903409 27996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:47.903386 27994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:47.904106 27982 server_base.cc:1034] running on GCE node
I20260812 07:14:47.904831 27982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:47.906138 27982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:47.907335 27982 hybrid_clock.cc:648] HybridClock initialized: now 1786518887907282 us; error 47 us; skew 500 ppm
I20260812 07:14:47.907657 27982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:47 dist-test-slave-lgr5 krb5kdc[27957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518887, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:47.910624 27982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:47.911909 27982 webserver.cc:460] Webserver started at http://127.0.0.1:45811/ using document root <none> and password file <none>
I20260812 07:14:47.912539 27982 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:47.912640 27982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:47.912885 27982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:47.915752 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "db7d2644d9ef43a39b5e6d55492949b3"
format_stamp: "Formatted at 2026-08-12 07:14:47 on dist-test-slave-lgr5"
server_key: "02443636f114dc6a47511eefe9744094"
server_key_iv: "c05e5b44a80fb8cd9b12c66902518711"
server_key_version: "encryptionkey@0"
I20260812 07:14:47.916493 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "db7d2644d9ef43a39b5e6d55492949b3"
format_stamp: "Formatted at 2026-08-12 07:14:47 on dist-test-slave-lgr5"
server_key: "02443636f114dc6a47511eefe9744094"
server_key_iv: "c05e5b44a80fb8cd9b12c66902518711"
server_key_version: "encryptionkey@0"
I20260812 07:14:47.921247 27982 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260812 07:14:47.924495 28010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:47.925734 27982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260812 07:14:47.925885 27982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "db7d2644d9ef43a39b5e6d55492949b3"
format_stamp: "Formatted at 2026-08-12 07:14:47 on dist-test-slave-lgr5"
server_key: "02443636f114dc6a47511eefe9744094"
server_key_iv: "c05e5b44a80fb8cd9b12c66902518711"
server_key_version: "encryptionkey@0"
I20260812 07:14:47.926054 27982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:47.931435 27982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:47.935061 27982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:47.935310 27982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:47.952198 27982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42685
I20260812 07:14:47.952221 28100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42685 every 8 connection(s)
I20260812 07:14:47.953452 27982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:47.956310 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 27982
I20260812 07:14:47.956503 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:47.957055 22229 external_mini_cluster.cc:1428] Setting key 286e1c1cdb3ef6406d7b34c5c35e6abe
I20260812 07:14:47.957230 28101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:47.964608 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:47.958669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46535 (local address 127.0.0.1:42685)
0812 07:14:47.959369 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:47.959379 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:47.959420 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:14:47.961571 (+ 2151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:47.961592 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:47.961882 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:47.962180 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:47.963160 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:47.963169 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:47.963180 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:47.963194 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:47.963253 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:47.963277 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:47.963446 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:47.963551 (+ 105us) server_negotiation.cc:299] Negotiation successful
0812 07:14:47.963713 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":318,"threads_started":1}
I20260812 07:14:47.967792 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:47.970034 28101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3: Bootstrap starting.
I20260812 07:14:47.973825 28101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:47.975188 28101 log.cc:824] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:47.977387 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:47.973882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54537 (local address 127.0.0.1:42685)
0812 07:14:47.974142 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:47.974147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:47.974771 (+ 624us) server_negotiation.cc:407] Connection header received
0812 07:14:47.974954 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:47.974963 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:47.975030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:47.975108 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:47.976479 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:47.976485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:47.976489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:47.976507 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:47.976551 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:47.976571 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:47.976662 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:47.976940 (+ 278us) server_negotiation.cc:299] Negotiation successful
0812 07:14:47.976999 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260812 07:14:47.978415 28101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3: No bootstrap required, opened a new log
I20260812 07:14:47.985571 28101 raft_consensus.cc:348] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d2644d9ef43a39b5e6d55492949b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42685 } }
I20260812 07:14:47.985882 28101 raft_consensus.cc:374] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:47.985967 28101 raft_consensus.cc:729] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db7d2644d9ef43a39b5e6d55492949b3, State: Initialized, Role: FOLLOWER
I20260812 07:14:47.986449 28101 consensus_queue.cc:260] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d2644d9ef43a39b5e6d55492949b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42685 } }
I20260812 07:14:47.986665 28101 raft_consensus.cc:388] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:47.986790 28101 raft_consensus.cc:482] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:47.986941 28101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:47.989221 28101 raft_consensus.cc:504] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d2644d9ef43a39b5e6d55492949b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42685 } }
I20260812 07:14:47.989661 28101 leader_election.cc:302] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db7d2644d9ef43a39b5e6d55492949b3; no voters:
I20260812 07:14:47.990172 28101 leader_election.cc:288] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:47.990276 28119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:47.990868 28119 raft_consensus.cc:686] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [term 1 LEADER]: Becoming Leader. State: Replica: db7d2644d9ef43a39b5e6d55492949b3, State: Running, Role: LEADER
I20260812 07:14:47.991391 28119 consensus_queue.cc:237] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d2644d9ef43a39b5e6d55492949b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42685 } }
I20260812 07:14:47.991911 28101 sys_catalog.cc:564] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:47.993790 28120 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db7d2644d9ef43a39b5e6d55492949b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d2644d9ef43a39b5e6d55492949b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42685 } } }
I20260812 07:14:47.994210 28120 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:47.994124 28122 sys_catalog.cc:455] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db7d2644d9ef43a39b5e6d55492949b3. Latest consensus state: current_term: 1 leader_uuid: "db7d2644d9ef43a39b5e6d55492949b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db7d2644d9ef43a39b5e6d55492949b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42685 } } }
I20260812 07:14:47.994390 28122 sys_catalog.cc:458] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:47.995314 28137 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:47.999534 28137 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:48.008282 28137 catalog_manager.cc:1347] Generated new cluster ID: 78e44f795d41405bb3bce6d0c4ada75b
I20260812 07:14:48.008376 28137 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:48.022035 28137 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:48.022404 28137 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:48.036288 28137 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db7d2644d9ef43a39b5e6d55492949b3: Generated new TSK 0
I20260812 07:14:48.037094 28137 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:48.051461 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27982
W20260812 07:14:48.052591 28096 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46535
W20260812 07:14:48.053249 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 27982: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:48.053419 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 27982
2026-08-12T07:14:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2016 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:48 dist-test-slave-lgr5 krb5kdc[28162](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:48 dist-test-slave-lgr5 krb5kdc[28162](info): set up 2 sockets
Aug 12 07:14:48 dist-test-slave-lgr5 krb5kdc[28162](info): commencing operation
krb5kdc: starting...
W20260812 07:14:49.316288 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:49 dist-test-slave-lgr5 krb5kdc[28162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:49.576505 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39583
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39583
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:49.903808 28196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:49.904107 28196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:49.904251 28196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:49.911232 28196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:49.911330 28196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:49.911404 28196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:49.911478 28196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:49.911535 28196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:49.921036 28196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:49.922149 28196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:49.923336 28196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:49.934158 28209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:49.934188 28207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:49.934399 28206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:49.935024 28196 server_base.cc:1034] running on GCE node
I20260812 07:14:49.935760 28196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:49.937095 28196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:49.938400 28196 hybrid_clock.cc:648] HybridClock initialized: now 1786518889938342 us; error 44 us; skew 500 ppm
I20260812 07:14:49.938750 28196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:49 dist-test-slave-lgr5 krb5kdc[28162](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:49.941665 28196 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:49.942989 28196 webserver.cc:460] Webserver started at http://127.0.0.1:39509/ using document root <none> and password file <none>
I20260812 07:14:49.943603 28196 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:49.943727 28196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:49.944007 28196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:49.946825 28196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "34b841a4bb15449ca1a08d04754ff776"
format_stamp: "Formatted at 2026-08-12 07:14:49 on dist-test-slave-lgr5"
server_key: "f533ae3ffcaa08e289686f13a70f16ac"
server_key_iv: "b974a0894717a360a14e87a689c1f976"
server_key_version: "encryptionkey@0"
I20260812 07:14:49.947610 28196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "34b841a4bb15449ca1a08d04754ff776"
format_stamp: "Formatted at 2026-08-12 07:14:49 on dist-test-slave-lgr5"
server_key: "f533ae3ffcaa08e289686f13a70f16ac"
server_key_iv: "b974a0894717a360a14e87a689c1f976"
server_key_version: "encryptionkey@0"
I20260812 07:14:49.952760 28196 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260812 07:14:49.957252 28224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:49.958738 28196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260812 07:14:49.958918 28196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "34b841a4bb15449ca1a08d04754ff776"
format_stamp: "Formatted at 2026-08-12 07:14:49 on dist-test-slave-lgr5"
server_key: "f533ae3ffcaa08e289686f13a70f16ac"
server_key_iv: "b974a0894717a360a14e87a689c1f976"
server_key_version: "encryptionkey@0"
I20260812 07:14:49.959115 28196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:49.990978 28196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:49.994879 28196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:49.995111 28196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:50.011082 28196 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39583
I20260812 07:14:50.011112 28314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39583 every 8 connection(s)
I20260812 07:14:50.012321 28196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:50.015780 28316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:50.018512 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28196
I20260812 07:14:50.018674 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:50.019097 22229 external_mini_cluster.cc:1428] Setting key df198415d68022c8a34245398d253c86
I20260812 07:14:50.027019 28316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776: Bootstrap starting.
I20260812 07:14:50.030653 28316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:50.031925 28316 log.cc:824] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776: Log is configured to *not* fsync() on all Append() calls
Aug 12 07:14:50 dist-test-slave-lgr5 krb5kdc[28162](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:14:50.035065 28316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776: No bootstrap required, opened a new log
I20260812 07:14:50.041450 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:50.020722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:47663 (local address 127.0.0.1:39583)
0812 07:14:50.021430 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:50.021441 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:50.021479 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:14:50.023081 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:50.023103 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:50.023455 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:50.023767 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:50.025462 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:50.026693 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:50.028448 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:50.029107 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:50.029226 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:50.034313 (+ 5087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:50.034320 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:50.034333 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:50.034371 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:50.037382 (+ 3011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:50.038242 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:50.038248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:50.038253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:50.038427 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:50.038873 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:50.038877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:50.038879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:50.039357 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:50.039600 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:50.040047 (+ 447us) server_negotiation.cc:299] Negotiation successful
0812 07:14:50.040374 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":319,"threads_started":1}
I20260812 07:14:50.043435 28316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b841a4bb15449ca1a08d04754ff776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39583 } }
I20260812 07:14:50.043788 28316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:50.043900 28316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b841a4bb15449ca1a08d04754ff776, State: Initialized, Role: FOLLOWER
I20260812 07:14:50.044445 28316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b841a4bb15449ca1a08d04754ff776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39583 } }
I20260812 07:14:50.044691 28316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:50.044832 28316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:50.044968 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:50.045007 28316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:50.047919 28316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b841a4bb15449ca1a08d04754ff776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39583 } }
I20260812 07:14:50.048525 28316 leader_election.cc:302] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34b841a4bb15449ca1a08d04754ff776; no voters:
I20260812 07:14:50.049154 28316 leader_election.cc:288] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:50.049260 28331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:50.051240 28316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:50.051342 28331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [term 1 LEADER]: Becoming Leader. State: Replica: 34b841a4bb15449ca1a08d04754ff776, State: Running, Role: LEADER
I20260812 07:14:50.051918 28331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b841a4bb15449ca1a08d04754ff776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39583 } }
I20260812 07:14:50.055310 28336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34b841a4bb15449ca1a08d04754ff776. Latest consensus state: current_term: 1 leader_uuid: "34b841a4bb15449ca1a08d04754ff776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b841a4bb15449ca1a08d04754ff776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39583 } } }
I20260812 07:14:50.055265 28334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34b841a4bb15449ca1a08d04754ff776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b841a4bb15449ca1a08d04754ff776" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39583 } } }
I20260812 07:14:50.055882 28336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:50.055884 28334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:50.060958 28353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:50.066309 28353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:50.066844 28324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:50.050652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:53055 (local address 127.0.0.1:39583)
0812 07:14:50.050808 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:50.050814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:50.050852 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:14:50.051165 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:50.051171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:50.051256 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:50.051334 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:50.053404 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:50.054745 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:50.056631 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:50.057368 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:50.057472 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:50.060218 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:50.060224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:50.060229 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:50.060265 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:50.063652 (+ 3387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:50.064520 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:50.064526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:50.064530 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:50.064670 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:50.065243 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:50.065249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:50.065256 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:50.065652 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:50.065901 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:50.066319 (+ 418us) server_negotiation.cc:299] Negotiation successful
0812 07:14:50.066497 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260812 07:14:50.075747 28353 catalog_manager.cc:1347] Generated new cluster ID: 3bcced2795394f929fb03f8e8b6dcd08
I20260812 07:14:50.075837 28353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:50.088796 28353 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:50.090428 28353 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:50.101444 28353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34b841a4bb15449ca1a08d04754ff776: Generated new TSK 0
I20260812 07:14:50.102156 28353 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:50.112614 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28196
I20260812 07:14:50.114933 28305 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:50.203024 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28196
W20260812 07:14:50.228731 28159 connection.cc:535] client connection to 127.0.0.1:39583 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39583): Connection reset by peer (error 104)
2026-08-12T07:14:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2145 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:50 dist-test-slave-lgr5 krb5kdc[28379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:50 dist-test-slave-lgr5 krb5kdc[28379](info): set up 2 sockets
Aug 12 07:14:50 dist-test-slave-lgr5 krb5kdc[28379](info): commencing operation
krb5kdc: starting...
W20260812 07:14:51.447729 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.003s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:51 dist-test-slave-lgr5 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:51.715337 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39503
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39503
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:52.033406 28404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:52.033648 28404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:52.033743 28404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:52.040580 28404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:52.040673 28404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:52.040724 28404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:52.040781 28404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:52.040823 28404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:52.050005 28404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39503
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:52.051056 28404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:52.052258 28404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:52.062903 28416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:52.062949 28413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:52.062852 28414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:52.063864 28404 server_base.cc:1034] running on GCE node
I20260812 07:14:52.064566 28404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:52.065759 28404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:52.066948 28404 hybrid_clock.cc:648] HybridClock initialized: now 1786518892066911 us; error 30 us; skew 500 ppm
I20260812 07:14:52.067239 28404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:52 dist-test-slave-lgr5 krb5kdc[28379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:52.070142 28404 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:52.071491 28404 webserver.cc:460] Webserver started at http://127.0.0.1:45335/ using document root <none> and password file <none>
I20260812 07:14:52.072067 28404 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:52.072206 28404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:52.072451 28404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:52.075153 28404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "2c252ff22c814b72b8c8d8e794ab702d"
format_stamp: "Formatted at 2026-08-12 07:14:52 on dist-test-slave-lgr5"
server_key: "9b7ad62ecba5f41881b4c186522fedc5"
server_key_iv: "216123deca57affff781a1d98374b704"
server_key_version: "encryptionkey@0"
I20260812 07:14:52.075842 28404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "2c252ff22c814b72b8c8d8e794ab702d"
format_stamp: "Formatted at 2026-08-12 07:14:52 on dist-test-slave-lgr5"
server_key: "9b7ad62ecba5f41881b4c186522fedc5"
server_key_iv: "216123deca57affff781a1d98374b704"
server_key_version: "encryptionkey@0"
I20260812 07:14:52.080533 28404 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260812 07:14:52.083663 28434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:52.084834 28404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:14:52.084972 28404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "2c252ff22c814b72b8c8d8e794ab702d"
format_stamp: "Formatted at 2026-08-12 07:14:52 on dist-test-slave-lgr5"
server_key: "9b7ad62ecba5f41881b4c186522fedc5"
server_key_iv: "216123deca57affff781a1d98374b704"
server_key_version: "encryptionkey@0"
I20260812 07:14:52.085130 28404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:52.097602 28404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:52.101392 28404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:52.101596 28404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:52.118170 28404 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39503
I20260812 07:14:52.118204 28535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39503 every 8 connection(s)
I20260812 07:14:52.119369 28404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:52.123375 28536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:52.125484 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28404
I20260812 07:14:52.125656 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:52.126194 22229 external_mini_cluster.cc:1428] Setting key b150fc04e18fde32ab9eebac7805c7ef
I20260812 07:14:52.134992 28536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d: Bootstrap starting.
I20260812 07:14:52.138334 28536 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:52.138878 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:52.127835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:36225 (local address 127.0.0.1:39503)
0812 07:14:52.128600 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:52.128614 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:52.128653 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:14:52.130283 (+ 1630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:52.130303 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:52.130656 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:52.130935 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:52.131540 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:52.132922 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:52.134695 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:52.135364 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:52.135506 (+ 142us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:52.136722 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:52.136729 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:52.136741 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:52.137216 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:52.137288 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:52.137312 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:52.137421 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:52.137701 (+ 280us) server_negotiation.cc:299] Negotiation successful
0812 07:14:52.137852 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":357,"threads_started":1}
I20260812 07:14:52.139424 28536 log.cc:824] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:52.142271 28536 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d: No bootstrap required, opened a new log
I20260812 07:14:52.142647 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:52.151564 28536 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c252ff22c814b72b8c8d8e794ab702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39503 } }
I20260812 07:14:52.151979 28536 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:52.152112 28536 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c252ff22c814b72b8c8d8e794ab702d, State: Initialized, Role: FOLLOWER
I20260812 07:14:52.152704 28536 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c252ff22c814b72b8c8d8e794ab702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39503 } }
I20260812 07:14:52.152978 28536 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:52.153110 28536 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:52.153285 28536 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:52.154076 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:52.147513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:58695 (local address 127.0.0.1:39503)
0812 07:14:52.147693 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:52.147699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:52.148170 (+ 471us) server_negotiation.cc:407] Connection header received
0812 07:14:52.148307 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:52.148313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:52.148386 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:52.148491 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:52.148928 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:52.150040 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:52.151381 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:52.152038 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:52.152182 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:52.153338 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:52.153344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:52.153348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:52.153368 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:52.153432 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:52.153457 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:52.153577 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:52.153699 (+ 122us) server_negotiation.cc:299] Negotiation successful
0812 07:14:52.153749 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:14:52.155980 28536 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c252ff22c814b72b8c8d8e794ab702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39503 } }
I20260812 07:14:52.156589 28536 leader_election.cc:302] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c252ff22c814b72b8c8d8e794ab702d; no voters:
I20260812 07:14:52.157182 28536 leader_election.cc:288] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:52.157336 28556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:52.157999 28556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [term 1 LEADER]: Becoming Leader. State: Replica: 2c252ff22c814b72b8c8d8e794ab702d, State: Running, Role: LEADER
I20260812 07:14:52.158509 28556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c252ff22c814b72b8c8d8e794ab702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39503 } }
I20260812 07:14:52.159394 28536 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:52.161365 28562 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c252ff22c814b72b8c8d8e794ab702d. Latest consensus state: current_term: 1 leader_uuid: "2c252ff22c814b72b8c8d8e794ab702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c252ff22c814b72b8c8d8e794ab702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39503 } } }
I20260812 07:14:52.161417 28559 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c252ff22c814b72b8c8d8e794ab702d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c252ff22c814b72b8c8d8e794ab702d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39503 } } }
I20260812 07:14:52.161748 28562 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:52.161751 28559 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:52.164878 28572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:52.169513 28572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:52.178889 28572 catalog_manager.cc:1347] Generated new cluster ID: 78fb6d75a74c47528b694a37207d8d91
I20260812 07:14:52.178983 28572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:52.192186 28572 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:52.193292 28572 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:52.201017 28572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c252ff22c814b72b8c8d8e794ab702d: Generated new TSK 0
I20260812 07:14:52.201683 28572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:52.207715 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28404
W20260812 07:14:52.209105 28529 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:36225
W20260812 07:14:52.209762 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28404: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:52.209887 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28404
W20260812 07:14:52.235451 28377 connection.cc:535] client connection to 127.0.0.1:39503 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39503): Connection reset by peer (error 104)
2026-08-12T07:14:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2006 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:52 dist-test-slave-lgr5 krb5kdc[28599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:52 dist-test-slave-lgr5 krb5kdc[28599](info): set up 2 sockets
Aug 12 07:14:52 dist-test-slave-lgr5 krb5kdc[28599](info): commencing operation
krb5kdc: starting...
W20260812 07:14:53.457510 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:53 dist-test-slave-lgr5 krb5kdc[28599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:53.720049 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:54.046314 28625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:54.046581 28625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:54.046720 28625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:54.053709 28625 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:54.053821 28625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:54.053896 28625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:54.053982 28625 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:54.054037 28625 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:54.063347 28625 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:54.064538 28625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:54.065718 28625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:54.076426 28638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:54.076382 28634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:54.076462 28635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:54.077162 28625 server_base.cc:1034] running on GCE node
I20260812 07:14:54.078049 28625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:54.079360 28625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:54.080560 28625 hybrid_clock.cc:648] HybridClock initialized: now 1786518894080518 us; error 33 us; skew 500 ppm
I20260812 07:14:54.080874 28625 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:54 dist-test-slave-lgr5 krb5kdc[28599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:54.083801 28625 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:54.085196 28625 webserver.cc:460] Webserver started at http://127.0.0.1:42395/ using document root <none> and password file <none>
I20260812 07:14:54.085837 28625 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:54.085956 28625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:54.086244 28625 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:54.089057 28625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "b03a7d997f054ba1a73193ca69ce294b"
format_stamp: "Formatted at 2026-08-12 07:14:54 on dist-test-slave-lgr5"
server_key: "7083b21a7f58fcce0a8f946bb22ccdd1"
server_key_iv: "087639776983a075214c8f81151d9b4b"
server_key_version: "encryptionkey@0"
I20260812 07:14:54.089784 28625 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "b03a7d997f054ba1a73193ca69ce294b"
format_stamp: "Formatted at 2026-08-12 07:14:54 on dist-test-slave-lgr5"
server_key: "7083b21a7f58fcce0a8f946bb22ccdd1"
server_key_iv: "087639776983a075214c8f81151d9b4b"
server_key_version: "encryptionkey@0"
I20260812 07:14:54.094527 28625 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260812 07:14:54.097755 28647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:54.098889 28625 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:14:54.099056 28625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "b03a7d997f054ba1a73193ca69ce294b"
format_stamp: "Formatted at 2026-08-12 07:14:54 on dist-test-slave-lgr5"
server_key: "7083b21a7f58fcce0a8f946bb22ccdd1"
server_key_iv: "087639776983a075214c8f81151d9b4b"
server_key_version: "encryptionkey@0"
I20260812 07:14:54.099222 28625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:54.104535 28625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:54.108258 28625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:54.108518 28625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:54.125116 28625 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41831
I20260812 07:14:54.125166 28746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41831 every 8 connection(s)
I20260812 07:14:54.126308 28625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:54.130306 28747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:54.132128 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28625
I20260812 07:14:54.132313 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:54.132819 22229 external_mini_cluster.cc:1428] Setting key 5aa998305572d6e420a5be419806e7fb
I20260812 07:14:54.139873 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:54.134469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:35209 (local address 127.0.0.1:41831)
0812 07:14:54.135202 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:54.135216 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:54.135261 (+ 45us) server_negotiation.cc:407] Connection header received
0812 07:14:54.136839 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:54.136861 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:54.137162 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:54.137469 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:54.138415 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:54.138421 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:54.138433 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:54.138446 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:54.138503 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:54.138523 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:54.138724 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:54.138834 (+ 110us) server_negotiation.cc:299] Negotiation successful
0812 07:14:54.139005 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":330,"threads_started":1}
I20260812 07:14:54.143039 28747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b: Bootstrap starting.
I20260812 07:14:54.143117 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:54.145879 28747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:54.146863 28747 log.cc:824] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:54.149505 28747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b: No bootstrap required, opened a new log
I20260812 07:14:54.149514 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:54.147187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:52593 (local address 127.0.0.1:41831)
0812 07:14:54.147358 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:54.147365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:54.147845 (+ 480us) server_negotiation.cc:407] Connection header received
0812 07:14:54.147984 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:54.147989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:54.148038 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:54.148129 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:54.148940 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:54.148945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:54.148950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:54.148968 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:54.149002 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:54.149016 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:54.149107 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:54.149215 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 07:14:54.149269 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260812 07:14:54.156839 28747 raft_consensus.cc:348] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03a7d997f054ba1a73193ca69ce294b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41831 } }
I20260812 07:14:54.157155 28747 raft_consensus.cc:374] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:54.157243 28747 raft_consensus.cc:729] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b03a7d997f054ba1a73193ca69ce294b, State: Initialized, Role: FOLLOWER
I20260812 07:14:54.157740 28747 consensus_queue.cc:260] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03a7d997f054ba1a73193ca69ce294b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41831 } }
I20260812 07:14:54.157949 28747 raft_consensus.cc:388] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:54.158070 28747 raft_consensus.cc:482] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:54.158218 28747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:54.160285 28747 raft_consensus.cc:504] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03a7d997f054ba1a73193ca69ce294b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41831 } }
I20260812 07:14:54.160734 28747 leader_election.cc:302] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b03a7d997f054ba1a73193ca69ce294b; no voters:
I20260812 07:14:54.161264 28747 leader_election.cc:288] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:54.161335 28767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:54.163132 28747 sys_catalog.cc:564] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:54.163507 28767 raft_consensus.cc:686] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [term 1 LEADER]: Becoming Leader. State: Replica: b03a7d997f054ba1a73193ca69ce294b, State: Running, Role: LEADER
I20260812 07:14:54.163985 28767 consensus_queue.cc:237] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03a7d997f054ba1a73193ca69ce294b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41831 } }
I20260812 07:14:54.166966 28770 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b03a7d997f054ba1a73193ca69ce294b. Latest consensus state: current_term: 1 leader_uuid: "b03a7d997f054ba1a73193ca69ce294b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03a7d997f054ba1a73193ca69ce294b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41831 } } }
I20260812 07:14:54.166980 28769 sys_catalog.cc:455] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b03a7d997f054ba1a73193ca69ce294b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b03a7d997f054ba1a73193ca69ce294b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41831 } } }
I20260812 07:14:54.167584 28769 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:54.168490 28770 sys_catalog.cc:458] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:54.175972 28794 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:54.180060 28794 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:54.188359 28794 catalog_manager.cc:1347] Generated new cluster ID: ee03647dcf824236ac2ca1db2383bf15
I20260812 07:14:54.188462 28794 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:54.211656 28794 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:54.212109 28794 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:54.225332 28794 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b03a7d997f054ba1a73193ca69ce294b: Generated new TSK 0
I20260812 07:14:54.226195 28794 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:54.235836 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28625
W20260812 07:14:54.236866 28741 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:35209
W20260812 07:14:54.237475 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28625: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:54.237600 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28625
2026-08-12T07:14:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2028 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:54 dist-test-slave-lgr5 krb5kdc[28808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:54 dist-test-slave-lgr5 krb5kdc[28808](info): set up 2 sockets
Aug 12 07:14:54 dist-test-slave-lgr5 krb5kdc[28808](info): commencing operation
krb5kdc: starting...
W20260812 07:14:55.516877 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.006s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:55 dist-test-slave-lgr5 krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:55.772850 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45755
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:56.102969 28829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:56.103246 28829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:56.103360 28829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:56.110402 28829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:56.110523 28829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:56.110599 28829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:56.110677 28829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:56.110726 28829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:56.120195 28829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45755
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:56.121276 28829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:56.122524 28829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:56.133687 28835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:56.133720 28840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:56.133687 28837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:56.134650 28829 server_base.cc:1034] running on GCE node
I20260812 07:14:56.135596 28829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:56.137262 28829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:56.138460 28829 hybrid_clock.cc:648] HybridClock initialized: now 1786518896138416 us; error 42 us; skew 500 ppm
I20260812 07:14:56.138787 28829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:56 dist-test-slave-lgr5 krb5kdc[28808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518896, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:56.144462 28829 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:56.145761 28829 webserver.cc:460] Webserver started at http://127.0.0.1:32849/ using document root <none> and password file <none>
I20260812 07:14:56.146363 28829 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:56.146477 28829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:56.146759 28829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:56.149631 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "ce7640b25b0b4d46b405436a6b3783f7"
format_stamp: "Formatted at 2026-08-12 07:14:56 on dist-test-slave-lgr5"
server_key: "b7e6c18835cc88da9e8b817992d10a4c"
server_key_iv: "d25e2c1dec67925bdd929193abc8d68c"
server_key_version: "encryptionkey@0"
I20260812 07:14:56.150362 28829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "ce7640b25b0b4d46b405436a6b3783f7"
format_stamp: "Formatted at 2026-08-12 07:14:56 on dist-test-slave-lgr5"
server_key: "b7e6c18835cc88da9e8b817992d10a4c"
server_key_iv: "d25e2c1dec67925bdd929193abc8d68c"
server_key_version: "encryptionkey@0"
I20260812 07:14:56.155149 28829 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:14:56.158478 28853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:56.159708 28829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260812 07:14:56.159858 28829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "ce7640b25b0b4d46b405436a6b3783f7"
format_stamp: "Formatted at 2026-08-12 07:14:56 on dist-test-slave-lgr5"
server_key: "b7e6c18835cc88da9e8b817992d10a4c"
server_key_iv: "d25e2c1dec67925bdd929193abc8d68c"
server_key_version: "encryptionkey@0"
I20260812 07:14:56.160019 28829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:56.181301 28829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:56.185402 28829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:56.185644 28829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:56.202344 28829 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45755
I20260812 07:14:56.202384 28939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45755 every 8 connection(s)
I20260812 07:14:56.203603 28829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:56.204295 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 28829
I20260812 07:14:56.204461 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:56.204985 22229 external_mini_cluster.cc:1428] Setting key 9dcceba21fe6a2f0b4a1ab53b8fb2066
I20260812 07:14:56.207681 28942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 12 07:14:56 dist-test-slave-lgr5 krb5kdc[28808](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260812 07:14:56.217940 28942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7: Bootstrap starting.
I20260812 07:14:56.221308 28942 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:56.222368 28942 log.cc:824] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:56.225358 28942 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7: No bootstrap required, opened a new log
I20260812 07:14:56.226114 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:56.206657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:41631 (local address 127.0.0.1:45755)
0812 07:14:56.207361 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:56.207373 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:56.207411 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:14:56.209072 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:56.209096 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:56.209438 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:56.209780 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:56.211261 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:56.212636 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:56.213892 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:56.214542 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:56.214720 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:56.219138 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:56.219145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:56.219158 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:56.219198 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:56.222069 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:56.222971 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:56.222979 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:56.222988 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:56.223195 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:56.223620 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:56.223623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:56.223626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:56.224151 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:56.224373 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:56.224873 (+ 500us) server_negotiation.cc:299] Negotiation successful
0812 07:14:56.225112 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":302,"threads_started":1}
I20260812 07:14:56.229831 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:56.233369 28942 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7640b25b0b4d46b405436a6b3783f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45755 } }
I20260812 07:14:56.233711 28942 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:56.233827 28942 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce7640b25b0b4d46b405436a6b3783f7, State: Initialized, Role: FOLLOWER
I20260812 07:14:56.234551 28942 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7640b25b0b4d46b405436a6b3783f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45755 } }
I20260812 07:14:56.234822 28942 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:56.234972 28942 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:56.235167 28942 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:56.237583 28942 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7640b25b0b4d46b405436a6b3783f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45755 } }
I20260812 07:14:56.238036 28942 leader_election.cc:302] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce7640b25b0b4d46b405436a6b3783f7; no voters:
I20260812 07:14:56.238623 28942 leader_election.cc:288] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:56.238722 28959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:56.239758 28959 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [term 1 LEADER]: Becoming Leader. State: Replica: ce7640b25b0b4d46b405436a6b3783f7, State: Running, Role: LEADER
I20260812 07:14:56.240285 28959 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7640b25b0b4d46b405436a6b3783f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45755 } }
I20260812 07:14:56.240727 28942 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:56.242841 28960 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce7640b25b0b4d46b405436a6b3783f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7640b25b0b4d46b405436a6b3783f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45755 } } }
I20260812 07:14:56.243230 28960 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:56.244843 28961 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce7640b25b0b4d46b405436a6b3783f7. Latest consensus state: current_term: 1 leader_uuid: "ce7640b25b0b4d46b405436a6b3783f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce7640b25b0b4d46b405436a6b3783f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45755 } } }
I20260812 07:14:56.245144 28961 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:56.247581 28964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:56.250460 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:56.233963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:59485 (local address 127.0.0.1:45755)
0812 07:14:56.234147 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:56.234152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:56.234464 (+ 312us) server_negotiation.cc:407] Connection header received
0812 07:14:56.234630 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:56.234633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:56.234689 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:56.234770 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:56.236507 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:56.237260 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:56.238556 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:56.239313 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:56.239428 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:56.242653 (+ 3225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:56.242659 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:56.242664 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:14:56.242704 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:56.246396 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:56.247298 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:56.247303 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:56.247307 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:56.247420 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:14:56.248243 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:14:56.248248 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:14:56.248255 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:14:56.248879 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:56.249211 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:56.249764 (+ 553us) server_negotiation.cc:299] Negotiation successful
0812 07:14:56.250046 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260812 07:14:56.259550 28964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:56.268584 28964 catalog_manager.cc:1347] Generated new cluster ID: 6cdcb78ee18d4570ae306780253caf27
I20260812 07:14:56.268680 28964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:56.277459 28964 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:56.278584 28964 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:56.283156 28964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce7640b25b0b4d46b405436a6b3783f7: Generated new TSK 0
I20260812 07:14:56.283807 28964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:56.295888 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 28829
I20260812 07:14:56.297495 28931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:14:56.384004 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 28829
W20260812 07:14:56.410847 28805 connection.cc:535] client connection to 127.0.0.1:45755 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45755): Connection reset by peer (error 104)
2026-08-12T07:14:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2146 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:56 dist-test-slave-lgr5 krb5kdc[28998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:56 dist-test-slave-lgr5 krb5kdc[28998](info): set up 2 sockets
Aug 12 07:14:56 dist-test-slave-lgr5 krb5kdc[28998](info): commencing operation
krb5kdc: starting...
W20260812 07:14:57.635896 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.206s user 0.012s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:57 dist-test-slave-lgr5 krb5kdc[28998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:57.900635 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:14:58.227432 29021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:14:58.227689 29021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:14:58.227818 29021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:14:58.234779 29021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:14:58.234879 29021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:14:58.234956 29021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:14:58.235069 29021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:14:58.235121 29021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:14:58.244325 29021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38087
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:14:58.245453 29021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:14:58.246619 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:14:58.257268 29033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:58.257369 29031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:14:58.257602 29028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:14:58.258275 29021 server_base.cc:1034] running on GCE node
I20260812 07:14:58.258935 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:14:58.260201 29021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:14:58.261394 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786518898261353 us; error 36 us; skew 500 ppm
I20260812 07:14:58.261698 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:14:58 dist-test-slave-lgr5 krb5kdc[28998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518898, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:14:58.264606 29021 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:14:58.265873 29021 webserver.cc:460] Webserver started at http://127.0.0.1:34911/ using document root <none> and password file <none>
I20260812 07:14:58.266481 29021 fs_manager.cc:359] Metadata directory not provided
I20260812 07:14:58.266600 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:14:58.266870 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:14:58.269683 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5"
format_stamp: "Formatted at 2026-08-12 07:14:58 on dist-test-slave-lgr5"
server_key: "262dedb3b47159710238689a16609074"
server_key_iv: "e48ba61d5d46dc31e7007edd8d1a1b04"
server_key_version: "encryptionkey@0"
I20260812 07:14:58.270386 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5"
format_stamp: "Formatted at 2026-08-12 07:14:58 on dist-test-slave-lgr5"
server_key: "262dedb3b47159710238689a16609074"
server_key_iv: "e48ba61d5d46dc31e7007edd8d1a1b04"
server_key_version: "encryptionkey@0"
I20260812 07:14:58.275099 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:14:58.278319 29043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:14:58.279500 29021 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:14:58.279656 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5"
format_stamp: "Formatted at 2026-08-12 07:14:58 on dist-test-slave-lgr5"
server_key: "262dedb3b47159710238689a16609074"
server_key_iv: "e48ba61d5d46dc31e7007edd8d1a1b04"
server_key_version: "encryptionkey@0"
I20260812 07:14:58.279836 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:14:58.301105 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:14:58.305037 29021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:14:58.305284 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:14:58.322237 29021 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38087
I20260812 07:14:58.322274 29159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38087 every 8 connection(s)
I20260812 07:14:58.323504 29021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:14:58.326928 29164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:14:58.328162 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29021
I20260812 07:14:58.328325 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:14:58.328737 22229 external_mini_cluster.cc:1428] Setting key 0c07c7999e5b735b281242b03c4aba5e
I20260812 07:14:58.340794 29164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5: Bootstrap starting.
I20260812 07:14:58.342592 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:58.330529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:56331 (local address 127.0.0.1:38087)
0812 07:14:58.331315 (+ 786us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:58.331329 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:58.331383 (+ 54us) server_negotiation.cc:407] Connection header received
0812 07:14:58.333150 (+ 1767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:58.333178 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:58.333632 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:58.334077 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:58.334570 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:58.336419 (+ 1849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:58.337733 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:58.338772 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:58.338939 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:58.339934 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:58.339944 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:58.339961 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:58.340642 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:58.340730 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:58.340762 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:58.340932 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:58.341201 (+ 269us) server_negotiation.cc:299] Negotiation successful
0812 07:14:58.341404 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":325,"threads_started":1}
I20260812 07:14:58.344203 29164 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5: Neither blocks nor log segments found. Creating new log.
I20260812 07:14:58.345326 29164 log.cc:824] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5: Log is configured to *not* fsync() on all Append() calls
I20260812 07:14:58.346119 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:14:58.348172 29164 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5: No bootstrap required, opened a new log
I20260812 07:14:58.357170 29168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:14:58.351102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:36341 (local address 127.0.0.1:38087)
0812 07:14:58.351275 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:14:58.351280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:14:58.351657 (+ 377us) server_negotiation.cc:407] Connection header received
0812 07:14:58.351801 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:14:58.351806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:14:58.351875 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:14:58.351984 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:14:58.352487 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:58.353592 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:58.354785 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:14:58.355455 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:14:58.355605 (+ 150us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:14:58.356533 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:14:58.356539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:14:58.356544 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:14:58.356563 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:14:58.356604 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:14:58.356621 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:14:58.356729 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0812 07:14:58.356844 (+ 115us) server_negotiation.cc:299] Negotiation successful
0812 07:14:58.356890 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:14:58.357268 29164 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260812 07:14:58.357632 29164 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:14:58.357733 29164 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe0344a87bcf4cb8a6fc8f4e6a7130d5, State: Initialized, Role: FOLLOWER
I20260812 07:14:58.358352 29164 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260812 07:14:58.358631 29164 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:14:58.358772 29164 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:14:58.358909 29164 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:14:58.361243 29164 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260812 07:14:58.361646 29164 leader_election.cc:302] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe0344a87bcf4cb8a6fc8f4e6a7130d5; no voters:
I20260812 07:14:58.362099 29164 leader_election.cc:288] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:14:58.362195 29181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:14:58.362818 29181 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [term 1 LEADER]: Becoming Leader. State: Replica: fe0344a87bcf4cb8a6fc8f4e6a7130d5, State: Running, Role: LEADER
I20260812 07:14:58.363296 29181 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } }
I20260812 07:14:58.363898 29164 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:14:58.365906 29182 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } } }
I20260812 07:14:58.366279 29182 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:58.366724 29183 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe0344a87bcf4cb8a6fc8f4e6a7130d5. Latest consensus state: current_term: 1 leader_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe0344a87bcf4cb8a6fc8f4e6a7130d5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38087 } } }
I20260812 07:14:58.367012 29183 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5 [sys.catalog]: This master's current role is: LEADER
I20260812 07:14:58.369172 29195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:14:58.373097 29195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:14:58.380528 29195 catalog_manager.cc:1347] Generated new cluster ID: 01c5cb2005324630b32cb866dcab3af1
I20260812 07:14:58.380621 29195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:14:58.410207 29195 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:14:58.411921 29195 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:14:58.425093 29195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe0344a87bcf4cb8a6fc8f4e6a7130d5: Generated new TSK 0
I20260812 07:14:58.426018 29195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:14:58.441011 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29021
W20260812 07:14:58.442132 29152 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:56331
W20260812 07:14:58.442867 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29021: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:14:58.442991 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29021
W20260812 07:14:58.464672 28993 connection.cc:535] client connection to 127.0.0.1:38087 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38087): Connection reset by peer (error 104)
2026-08-12T07:14:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:14:58 dist-test-slave-lgr5 krb5kdc[29232](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:14:58 dist-test-slave-lgr5 krb5kdc[29232](info): set up 2 sockets
Aug 12 07:14:58 dist-test-slave-lgr5 krb5kdc[29232](info): commencing operation
krb5kdc: starting...
W20260812 07:14:59.692920 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.002s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:14:59 dist-test-slave-lgr5 krb5kdc[29232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:14:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:14:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260812 07:14:59.975558 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43617
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43617
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:00.301201 29259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:00.301486 29259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:00.301630 29259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:00.308624 29259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:00.308736 29259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:00.308809 29259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:00.308900 29259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:00.308959 29259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:00.318454 29259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43617
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:00.319624 29259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:00.320901 29259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:00.331790 29272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:00.331751 29274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:00.331816 29277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:00.332993 29259 server_base.cc:1034] running on GCE node
I20260812 07:15:00.333693 29259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:00.334986 29259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:00.336166 29259 hybrid_clock.cc:648] HybridClock initialized: now 1786518900336109 us; error 49 us; skew 500 ppm
I20260812 07:15:00.336493 29259 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:00 dist-test-slave-lgr5 krb5kdc[29232](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518900, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:00.339418 29259 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260812 07:15:00.340765 29259 webserver.cc:460] Webserver started at http://127.0.0.1:40651/ using document root <none> and password file <none>
I20260812 07:15:00.341399 29259 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:00.341516 29259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:00.341797 29259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:00.344662 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "6580569e3227447eb21161fbdec472b9"
format_stamp: "Formatted at 2026-08-12 07:15:00 on dist-test-slave-lgr5"
server_key: "dd7013c582c5857da8ec57805bcb73f4"
server_key_iv: "d39f04d2fd67e5f58616ad9418bf4423"
server_key_version: "encryptionkey@0"
I20260812 07:15:00.345381 29259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "6580569e3227447eb21161fbdec472b9"
format_stamp: "Formatted at 2026-08-12 07:15:00 on dist-test-slave-lgr5"
server_key: "dd7013c582c5857da8ec57805bcb73f4"
server_key_iv: "d39f04d2fd67e5f58616ad9418bf4423"
server_key_version: "encryptionkey@0"
I20260812 07:15:00.350545 29259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260812 07:15:00.353789 29290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:00.355010 29259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260812 07:15:00.355180 29259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "6580569e3227447eb21161fbdec472b9"
format_stamp: "Formatted at 2026-08-12 07:15:00 on dist-test-slave-lgr5"
server_key: "dd7013c582c5857da8ec57805bcb73f4"
server_key_iv: "d39f04d2fd67e5f58616ad9418bf4423"
server_key_version: "encryptionkey@0"
I20260812 07:15:00.355360 29259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:00.360764 29259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:00.364317 29259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:00.364609 29259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:00.381250 29259 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43617
I20260812 07:15:00.381302 29381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43617 every 8 connection(s)
I20260812 07:15:00.382507 29259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:00.385905 29383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:00.391148 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29259
I20260812 07:15:00.391330 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:00.391882 22229 external_mini_cluster.cc:1428] Setting key f75a39efa8efaf5782c67daa71e159de
I20260812 07:15:00.397192 29383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9: Bootstrap starting.
I20260812 07:15:00.400811 29383 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:00.401854 29383 log.cc:824] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:00.404806 29383 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9: No bootstrap required, opened a new log
I20260812 07:15:00.405233 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:00.399163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:51351 (local address 127.0.0.1:43617)
0812 07:15:00.399926 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:00.399937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:00.399978 (+ 41us) server_negotiation.cc:407] Connection header received
0812 07:15:00.401760 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:00.401783 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:00.402120 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:00.402487 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:00.403723 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:00.403729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:00.403742 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:00.403757 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:00.403812 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:00.403835 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:00.403972 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:00.404178 (+ 206us) server_negotiation.cc:299] Negotiation successful
0812 07:15:00.404358 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":353,"threads_started":1}
I20260812 07:15:00.408846 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:00.412840 29383 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6580569e3227447eb21161fbdec472b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43617 } }
I20260812 07:15:00.413221 29383 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:00.413319 29383 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6580569e3227447eb21161fbdec472b9, State: Initialized, Role: FOLLOWER
I20260812 07:15:00.413888 29383 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6580569e3227447eb21161fbdec472b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43617 } }
I20260812 07:15:00.414117 29383 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:00.414245 29383 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:00.414417 29383 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:00.415318 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:00.412886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.158:57511 (local address 127.0.0.1:43617)
0812 07:15:00.413090 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:00.413096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:00.413558 (+ 462us) server_negotiation.cc:407] Connection header received
0812 07:15:00.413710 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:00.413714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:00.413759 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:00.413835 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:00.414708 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:00.414714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:00.414717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:00.414730 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:00.414767 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:00.414782 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:00.414893 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:00.414982 (+ 89us) server_negotiation.cc:299] Negotiation successful
0812 07:15:00.415044 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260812 07:15:00.417303 29383 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6580569e3227447eb21161fbdec472b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43617 } }
I20260812 07:15:00.417797 29383 leader_election.cc:302] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6580569e3227447eb21161fbdec472b9; no voters:
I20260812 07:15:00.418331 29383 leader_election.cc:288] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:00.418418 29402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:00.419054 29402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [term 1 LEADER]: Becoming Leader. State: Replica: 6580569e3227447eb21161fbdec472b9, State: Running, Role: LEADER
I20260812 07:15:00.419543 29402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6580569e3227447eb21161fbdec472b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43617 } }
I20260812 07:15:00.420251 29383 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:00.422150 29403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6580569e3227447eb21161fbdec472b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6580569e3227447eb21161fbdec472b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43617 } } }
I20260812 07:15:00.422549 29403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:00.424264 29405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6580569e3227447eb21161fbdec472b9. Latest consensus state: current_term: 1 leader_uuid: "6580569e3227447eb21161fbdec472b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6580569e3227447eb21161fbdec472b9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43617 } } }
I20260812 07:15:00.424504 29405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:00.425010 29412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:00.429872 29412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:00.438726 29412 catalog_manager.cc:1347] Generated new cluster ID: 3af2fa54c9fc4b8da137167fd5f7adbd
I20260812 07:15:00.438828 29412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:00.451105 29412 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:00.451469 29412 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:00.456909 29412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6580569e3227447eb21161fbdec472b9: Generated new TSK 0
I20260812 07:15:00.457710 29412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:00.469012 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29259
W20260812 07:15:00.470000 29373 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.158:51351
W20260812 07:15:00.470644 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29259: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:15:00.470773 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29259
2026-08-12T07:15:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2030 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:00 dist-test-slave-lgr5 krb5kdc[29446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:00 dist-test-slave-lgr5 krb5kdc[29446](info): set up 2 sockets
Aug 12 07:15:00 dist-test-slave-lgr5 krb5kdc[29446](info): commencing operation
krb5kdc: starting...
W20260812 07:15:01.726702 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:01 dist-test-slave-lgr5 krb5kdc[29446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:01.990638 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37413
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:36401
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:37413
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:02.310745 29462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:02.311002 29462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:02.311097 29462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:02.317958 29462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:02.318053 29462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:02.318104 29462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:02.318162 29462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:02.318204 29462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:02.327332 29462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:36401
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:37413
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:37413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:02.328411 29462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:02.329526 29462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:02.340461 29473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:02.340637 29475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:02.340612 29472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:02.341470 29462 server_base.cc:1034] running on GCE node
I20260812 07:15:02.342173 29462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:02.343422 29462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:02.344611 29462 hybrid_clock.cc:648] HybridClock initialized: now 1786518902344579 us; error 26 us; skew 500 ppm
I20260812 07:15:02.344892 29462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:02 dist-test-slave-lgr5 krb5kdc[29446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518902, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:02.351358 29462 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:02.352828 29462 webserver.cc:460] Webserver started at http://127.21.181.126:44181/ using document root <none> and password file <none>
I20260812 07:15:02.353417 29462 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:02.353516 29462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:02.353757 29462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:02.356529 29462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150"
format_stamp: "Formatted at 2026-08-12 07:15:02 on dist-test-slave-lgr5"
server_key: "66c20d530dc08cfa3bf495f355100086"
server_key_iv: "91b80d220fff2242d08555ec3664b892"
server_key_version: "encryptionkey@0"
I20260812 07:15:02.357213 29462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150"
format_stamp: "Formatted at 2026-08-12 07:15:02 on dist-test-slave-lgr5"
server_key: "66c20d530dc08cfa3bf495f355100086"
server_key_iv: "91b80d220fff2242d08555ec3664b892"
server_key_version: "encryptionkey@0"
I20260812 07:15:02.361809 29462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260812 07:15:02.364962 29488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:02.366149 29462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:15:02.366302 29462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150"
format_stamp: "Formatted at 2026-08-12 07:15:02 on dist-test-slave-lgr5"
server_key: "66c20d530dc08cfa3bf495f355100086"
server_key_iv: "91b80d220fff2242d08555ec3664b892"
server_key_version: "encryptionkey@0"
I20260812 07:15:02.366466 29462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:02.387295 29462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:02.391100 29462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:02.391331 29462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:02.408000 29462 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:37413
I20260812 07:15:02.408022 29563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:37413 every 8 connection(s)
I20260812 07:15:02.409199 29462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:02.412585 29568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:02.413508 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29462
I20260812 07:15:02.413702 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:02.414248 22229 external_mini_cluster.cc:1428] Setting key 4ce8277927eaa6d011debfd97f3a2aac
I20260812 07:15:02.425743 29568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150: Bootstrap starting.
Aug 12 07:15:02 dist-test-slave-lgr5 krb5kdc[29446](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:02.429117 29568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:02.430189 29568 log.cc:824] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:02.432752 29568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150: No bootstrap required, opened a new log
I20260812 07:15:02.436058 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:02.415952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36867 (local address 127.21.181.126:37413)
0812 07:15:02.416716 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:02.416727 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:02.416771 (+ 44us) server_negotiation.cc:407] Connection header received
0812 07:15:02.418496 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:02.418518 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:02.418878 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:02.419201 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:02.420874 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:02.422114 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:02.423843 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:02.424536 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:02.424658 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:02.429228 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:02.429239 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:02.429256 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:02.429302 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:02.432197 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:02.432935 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:02.432943 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:02.432948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:02.433109 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:02.433547 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:02.433550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:02.433552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:02.434041 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:02.434367 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:02.434840 (+ 473us) server_negotiation.cc:299] Negotiation successful
0812 07:15:02.435152 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":361,"threads_started":1}
I20260812 07:15:02.439826 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:02.440868 29568 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37413 } }
I20260812 07:15:02.441184 29568 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:02.441288 29568 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d81e6ebb69f46d4a4bbb912eeeb5150, State: Initialized, Role: FOLLOWER
I20260812 07:15:02.441793 29568 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37413 } }
I20260812 07:15:02.442010 29568 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:02.442132 29568 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:02.442291 29568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:02.444702 29568 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37413 } }
I20260812 07:15:02.445154 29568 leader_election.cc:302] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d81e6ebb69f46d4a4bbb912eeeb5150; no voters:
I20260812 07:15:02.445736 29568 leader_election.cc:288] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:02.445817 29585 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:02.447826 29568 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:02.448338 29585 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [term 1 LEADER]: Becoming Leader. State: Replica: 0d81e6ebb69f46d4a4bbb912eeeb5150, State: Running, Role: LEADER
I20260812 07:15:02.448866 29585 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37413 } }
I20260812 07:15:02.451679 29586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37413 } } }
I20260812 07:15:02.451750 29587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d81e6ebb69f46d4a4bbb912eeeb5150. Latest consensus state: current_term: 1 leader_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d81e6ebb69f46d4a4bbb912eeeb5150" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 37413 } } }
I20260812 07:15:02.452121 29586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:02.452121 29587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:02.453454 29605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:02.458535 29605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:02.469331 29574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:02.444137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55815 (local address 127.21.181.126:37413)
0812 07:15:02.444304 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:02.444310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:02.450643 (+ 6333us) server_negotiation.cc:407] Connection header received
0812 07:15:02.450845 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:02.450851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:02.450933 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:02.451017 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:02.453269 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:02.454390 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:02.456178 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:02.456761 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:02.456881 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:02.459203 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:02.459222 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:02.459228 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:02.459268 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:02.464927 (+ 5659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:02.466142 (+ 1215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:02.466154 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:02.466159 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:02.466376 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:02.467056 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:02.467063 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:02.467067 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:02.467535 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:02.467817 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:02.468667 (+ 850us) server_negotiation.cc:299] Negotiation successful
0812 07:15:02.468861 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:15:02.478367 29605 catalog_manager.cc:1347] Generated new cluster ID: dc2cb5281ec24c6c8933a07bf895dd49
I20260812 07:15:02.478641 29605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:02.494716 29605 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:02.496042 29605 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:02.506457 29605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d81e6ebb69f46d4a4bbb912eeeb5150: Generated new TSK 0
I20260812 07:15:02.507364 29605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:02.521251 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29462
I20260812 07:15:02.522256 29557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:02.611697 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29462
W20260812 07:15:02.642339 29443 connection.cc:535] client connection to 127.21.181.126:37413 recv error: Network error: failed to read from TLS socket (remote: 127.21.181.126:37413): Connection reset by peer (error 104)
2026-08-12T07:15:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2147 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:02 dist-test-slave-lgr5 krb5kdc[29635](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:02 dist-test-slave-lgr5 krb5kdc[29635](info): set up 2 sockets
Aug 12 07:15:02 dist-test-slave-lgr5 krb5kdc[29635](info): commencing operation
krb5kdc: starting...
W20260812 07:15:03.862057 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.202s user 0.001s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:04 dist-test-slave-lgr5 krb5kdc[29635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:04.136667 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38169
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:38169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:04.460563 29667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:04.460834 29667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:04.460973 29667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:04.467818 29667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:04.467924 29667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:04.468003 29667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:04.468108 29667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:04.468168 29667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:04.477334 29667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:38169
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:38169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:04.478459 29667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:04.479648 29667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:04.490581 29683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:04.490664 29678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:04.490621 29679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:04.491544 29667 server_base.cc:1034] running on GCE node
I20260812 07:15:04.492249 29667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:04.493500 29667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:04.494722 29667 hybrid_clock.cc:648] HybridClock initialized: now 1786518904494668 us; error 38 us; skew 500 ppm
I20260812 07:15:04.495072 29667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:04 dist-test-slave-lgr5 krb5kdc[29635](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518904, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:04.498214 29667 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:04.499491 29667 webserver.cc:460] Webserver started at http://127.21.181.126:42093/ using document root <none> and password file <none>
I20260812 07:15:04.500095 29667 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:04.500212 29667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:04.500453 29667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:04.503197 29667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "64961ade882f4e1b8e9699910ac88885"
format_stamp: "Formatted at 2026-08-12 07:15:04 on dist-test-slave-lgr5"
server_key: "77e94c60ae51bbe3735185f753b01342"
server_key_iv: "bcbc8bbc09426288845154f8b24fb5fb"
server_key_version: "encryptionkey@0"
I20260812 07:15:04.503870 29667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "64961ade882f4e1b8e9699910ac88885"
format_stamp: "Formatted at 2026-08-12 07:15:04 on dist-test-slave-lgr5"
server_key: "77e94c60ae51bbe3735185f753b01342"
server_key_iv: "bcbc8bbc09426288845154f8b24fb5fb"
server_key_version: "encryptionkey@0"
I20260812 07:15:04.508754 29667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260812 07:15:04.511895 29694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:04.513129 29667 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260812 07:15:04.513267 29667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "64961ade882f4e1b8e9699910ac88885"
format_stamp: "Formatted at 2026-08-12 07:15:04 on dist-test-slave-lgr5"
server_key: "77e94c60ae51bbe3735185f753b01342"
server_key_iv: "bcbc8bbc09426288845154f8b24fb5fb"
server_key_version: "encryptionkey@0"
I20260812 07:15:04.513419 29667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:04.529016 29667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:04.532698 29667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:04.532918 29667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:04.549530 29667 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:38169
I20260812 07:15:04.549559 29794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:38169 every 8 connection(s)
I20260812 07:15:04.550760 29667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:04.554265 29795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:04.558542 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29667
I20260812 07:15:04.558691 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:04.559161 22229 external_mini_cluster.cc:1428] Setting key 5dc3664a847b91c9597bafdd799a3968
I20260812 07:15:04.566218 29795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885: Bootstrap starting.
I20260812 07:15:04.569664 29795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:04.570875 29795 log.cc:824] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:04.573801 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:04.560869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48757 (local address 127.21.181.126:38169)
0812 07:15:04.561653 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:04.561670 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:04.561751 (+ 81us) server_negotiation.cc:407] Connection header received
0812 07:15:04.564517 (+ 2766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:04.564545 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:04.565001 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:04.565407 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:04.565889 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:04.567796 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:04.569110 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:04.570136 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:04.570322 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:04.571369 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:04.571380 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:04.571397 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:04.571933 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:04.572033 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:04.572065 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:04.572239 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:04.572461 (+ 222us) server_negotiation.cc:299] Negotiation successful
0812 07:15:04.572623 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":322,"threads_started":1}
I20260812 07:15:04.573933 29795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885: No bootstrap required, opened a new log
I20260812 07:15:04.577478 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:04.581748 29795 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64961ade882f4e1b8e9699910ac88885" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38169 } }
I20260812 07:15:04.582140 29795 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:04.582253 29795 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64961ade882f4e1b8e9699910ac88885, State: Initialized, Role: FOLLOWER
I20260812 07:15:04.582898 29795 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64961ade882f4e1b8e9699910ac88885" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38169 } }
I20260812 07:15:04.583199 29795 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:04.583338 29795 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:04.583513 29795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:04.586274 29795 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64961ade882f4e1b8e9699910ac88885" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38169 } }
I20260812 07:15:04.586833 29795 leader_election.cc:302] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64961ade882f4e1b8e9699910ac88885; no voters:
I20260812 07:15:04.587427 29795 leader_election.cc:288] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:04.587534 29815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:04.588403 29799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:04.581478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54507 (local address 127.21.181.126:38169)
0812 07:15:04.581611 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:04.581616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:04.582046 (+ 430us) server_negotiation.cc:407] Connection header received
0812 07:15:04.582232 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:04.582238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:04.582312 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:04.582409 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:04.583077 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:04.583928 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:04.585737 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:04.586157 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:04.586246 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:04.587381 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:04.587385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:04.587388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:04.587404 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:04.587464 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:04.587480 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:04.587593 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:04.588016 (+ 423us) server_negotiation.cc:299] Negotiation successful
0812 07:15:04.588101 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:15:04.589145 29815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [term 1 LEADER]: Becoming Leader. State: Replica: 64961ade882f4e1b8e9699910ac88885, State: Running, Role: LEADER
I20260812 07:15:04.589633 29795 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:04.589742 29815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64961ade882f4e1b8e9699910ac88885" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38169 } }
I20260812 07:15:04.592579 29818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64961ade882f4e1b8e9699910ac88885. Latest consensus state: current_term: 1 leader_uuid: "64961ade882f4e1b8e9699910ac88885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64961ade882f4e1b8e9699910ac88885" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38169 } } }
I20260812 07:15:04.592664 29816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64961ade882f4e1b8e9699910ac88885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64961ade882f4e1b8e9699910ac88885" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 38169 } } }
I20260812 07:15:04.592941 29818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:04.593015 29816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:04.594553 29830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:04.599902 29830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:04.608528 29830 catalog_manager.cc:1347] Generated new cluster ID: 5b70ac9cc4704222bcb6406b2ea7cdbc
I20260812 07:15:04.608620 29830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:04.620042 29830 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:04.621244 29830 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:04.629197 29830 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64961ade882f4e1b8e9699910ac88885: Generated new TSK 0
I20260812 07:15:04.629877 29830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:04.635478 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29667
W20260812 07:15:04.636584 29785 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48757
W20260812 07:15:04.637301 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29667: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:15:04.637449 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29667
W20260812 07:15:04.663504 29631 connection.cc:535] client connection to 127.21.181.126:38169 recv error: Network error: failed to read from TLS socket (remote: 127.21.181.126:38169): Connection reset by peer (error 104)
2026-08-12T07:15:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2021 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:04 dist-test-slave-lgr5 krb5kdc[29861](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:04 dist-test-slave-lgr5 krb5kdc[29861](info): set up 2 sockets
Aug 12 07:15:04 dist-test-slave-lgr5 krb5kdc[29861](info): commencing operation
krb5kdc: starting...
W20260812 07:15:05.903358 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:06 dist-test-slave-lgr5 krb5kdc[29861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:06.175478 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45631
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:45631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:06.495926 29882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:06.496204 29882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:06.496338 29882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:06.503247 29882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:06.503347 29882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:06.503423 29882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:06.503515 29882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:06.503571 29882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:06.512909 29882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:45631
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:06.514012 29882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:06.515260 29882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:06.526291 29891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:06.526211 29892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:06.526247 29895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:06.527055 29882 server_base.cc:1034] running on GCE node
I20260812 07:15:06.527788 29882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:06.529075 29882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:06.530275 29882 hybrid_clock.cc:648] HybridClock initialized: now 1786518906530225 us; error 41 us; skew 500 ppm
I20260812 07:15:06.530596 29882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:06 dist-test-slave-lgr5 krb5kdc[29861](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518906, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:06.533525 29882 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:06.534792 29882 webserver.cc:460] Webserver started at http://127.21.181.126:44063/ using document root <none> and password file <none>
I20260812 07:15:06.535395 29882 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:06.535491 29882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:06.535729 29882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:06.538540 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "c81e0dd533274072b0eedec09f725511"
format_stamp: "Formatted at 2026-08-12 07:15:06 on dist-test-slave-lgr5"
server_key: "b16d5cebb42d14a7de8e1da929d1154d"
server_key_iv: "9cf138a477660abdd91cc1f0db6c1f25"
server_key_version: "encryptionkey@0"
I20260812 07:15:06.539233 29882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "c81e0dd533274072b0eedec09f725511"
format_stamp: "Formatted at 2026-08-12 07:15:06 on dist-test-slave-lgr5"
server_key: "b16d5cebb42d14a7de8e1da929d1154d"
server_key_iv: "9cf138a477660abdd91cc1f0db6c1f25"
server_key_version: "encryptionkey@0"
I20260812 07:15:06.543810 29882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260812 07:15:06.546926 29906 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:06.548172 29882 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260812 07:15:06.548301 29882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "c81e0dd533274072b0eedec09f725511"
format_stamp: "Formatted at 2026-08-12 07:15:06 on dist-test-slave-lgr5"
server_key: "b16d5cebb42d14a7de8e1da929d1154d"
server_key_iv: "9cf138a477660abdd91cc1f0db6c1f25"
server_key_version: "encryptionkey@0"
I20260812 07:15:06.548507 29882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:06.553483 29882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:06.556898 29882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:06.557132 29882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:06.573172 29882 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:45631
I20260812 07:15:06.573207 30001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:45631 every 8 connection(s)
I20260812 07:15:06.574362 29882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:06.577754 30002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:06.578472 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 29882
I20260812 07:15:06.578652 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:06.579226 22229 external_mini_cluster.cc:1428] Setting key 9b4776c19e073e8df4a4378303fb3f67
I20260812 07:15:06.586607 30006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:06.580950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57749 (local address 127.21.181.126:45631)
0812 07:15:06.581730 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:06.581740 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:06.581779 (+ 39us) server_negotiation.cc:407] Connection header received
0812 07:15:06.583630 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:06.583653 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:06.583952 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:06.584300 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:06.585264 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:06.585270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:06.585282 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:06.585294 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:06.585347 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:06.585369 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:06.585533 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:06.585625 (+ 92us) server_negotiation.cc:299] Negotiation successful
0812 07:15:06.585777 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":324,"threads_started":1}
I20260812 07:15:06.590081 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:06.590548 30002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511: Bootstrap starting.
I20260812 07:15:06.593355 30002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:06.594434 30002 log.cc:824] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:06.596426 30006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:06.594183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34507 (local address 127.21.181.126:45631)
0812 07:15:06.594348 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:06.594353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:06.594725 (+ 372us) server_negotiation.cc:407] Connection header received
0812 07:15:06.594881 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:06.594885 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:06.594945 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:06.595045 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:06.595825 (+ 780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:06.595830 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:06.595834 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:06.595851 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:06.595891 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:06.595909 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:06.596032 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:06.596074 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 07:15:06.596170 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:15:06.597390 30002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511: No bootstrap required, opened a new log
I20260812 07:15:06.604709 30002 raft_consensus.cc:348] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c81e0dd533274072b0eedec09f725511" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45631 } }
I20260812 07:15:06.605096 30002 raft_consensus.cc:374] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:06.605173 30002 raft_consensus.cc:729] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c81e0dd533274072b0eedec09f725511, State: Initialized, Role: FOLLOWER
I20260812 07:15:06.605643 30002 consensus_queue.cc:260] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c81e0dd533274072b0eedec09f725511" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45631 } }
I20260812 07:15:06.605842 30002 raft_consensus.cc:388] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:06.605935 30002 raft_consensus.cc:482] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:06.606045 30002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:06.607932 30002 raft_consensus.cc:504] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c81e0dd533274072b0eedec09f725511" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45631 } }
I20260812 07:15:06.608385 30002 leader_election.cc:302] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c81e0dd533274072b0eedec09f725511; no voters:
I20260812 07:15:06.608888 30002 leader_election.cc:288] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:06.609004 30023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:06.610008 30023 raft_consensus.cc:686] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [term 1 LEADER]: Becoming Leader. State: Replica: c81e0dd533274072b0eedec09f725511, State: Running, Role: LEADER
I20260812 07:15:06.610488 30023 consensus_queue.cc:237] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c81e0dd533274072b0eedec09f725511" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45631 } }
I20260812 07:15:06.610736 30002 sys_catalog.cc:564] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:06.612941 30025 sys_catalog.cc:455] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c81e0dd533274072b0eedec09f725511. Latest consensus state: current_term: 1 leader_uuid: "c81e0dd533274072b0eedec09f725511" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c81e0dd533274072b0eedec09f725511" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45631 } } }
I20260812 07:15:06.613029 30024 sys_catalog.cc:455] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c81e0dd533274072b0eedec09f725511" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c81e0dd533274072b0eedec09f725511" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 45631 } } }
I20260812 07:15:06.613273 30025 sys_catalog.cc:458] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:06.613381 30024 sys_catalog.cc:458] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:06.614611 30038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:06.620646 30038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:06.629733 30038 catalog_manager.cc:1347] Generated new cluster ID: aabf53b42e3a48cf95a96b94e08bd847
I20260812 07:15:06.629823 30038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:06.664062 30038 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:06.664467 30038 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:06.675592 30038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c81e0dd533274072b0eedec09f725511: Generated new TSK 0
I20260812 07:15:06.676329 30038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:06.681226 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29882
W20260812 07:15:06.682163 29992 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57749
W20260812 07:15:06.682726 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 29882: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:15:06.682875 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 29882
2026-08-12T07:15:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:06 dist-test-slave-lgr5 krb5kdc[30058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:06 dist-test-slave-lgr5 krb5kdc[30058](info): set up 2 sockets
Aug 12 07:15:06 dist-test-slave-lgr5 krb5kdc[30058](info): commencing operation
krb5kdc: starting...
W20260812 07:15:07.926106 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:08 dist-test-slave-lgr5 krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:08.211360 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43369
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:42627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:08.531385 30088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:08.531648 30088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:08.531782 30088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:08.538684 30088 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:08.538784 30088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:08.538865 30088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:08.538944 30088 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:08.539000 30088 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:08.548246 30088 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:42627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:08.549382 30088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:08.550571 30088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:08.561326 30095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:08.561385 30096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:08.561412 30099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:08.562098 30088 server_base.cc:1034] running on GCE node
I20260812 07:15:08.562783 30088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:08.564070 30088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:08.565306 30088 hybrid_clock.cc:648] HybridClock initialized: now 1786518908565257 us; error 39 us; skew 500 ppm
I20260812 07:15:08.565631 30088 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:08 dist-test-slave-lgr5 krb5kdc[30058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518908, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:08.572606 30088 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:08.573910 30088 webserver.cc:460] Webserver started at http://127.21.181.126:40705/ using document root <none> and password file <none>
I20260812 07:15:08.574501 30088 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:08.574599 30088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:08.574831 30088 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:08.577615 30088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "515faed6f30b4649b5a2fa78d32da2bd"
format_stamp: "Formatted at 2026-08-12 07:15:08 on dist-test-slave-lgr5"
server_key: "45ee30de5894cad4024ba7383384ac02"
server_key_iv: "9259087ec0df7d752f139432366eccbe"
server_key_version: "encryptionkey@0"
I20260812 07:15:08.578276 30088 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "515faed6f30b4649b5a2fa78d32da2bd"
format_stamp: "Formatted at 2026-08-12 07:15:08 on dist-test-slave-lgr5"
server_key: "45ee30de5894cad4024ba7383384ac02"
server_key_iv: "9259087ec0df7d752f139432366eccbe"
server_key_version: "encryptionkey@0"
I20260812 07:15:08.582909 30088 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:15:08.586100 30119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:08.587248 30088 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:15:08.587414 30088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "515faed6f30b4649b5a2fa78d32da2bd"
format_stamp: "Formatted at 2026-08-12 07:15:08 on dist-test-slave-lgr5"
server_key: "45ee30de5894cad4024ba7383384ac02"
server_key_iv: "9259087ec0df7d752f139432366eccbe"
server_key_version: "encryptionkey@0"
I20260812 07:15:08.587580 30088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:08.604278 30088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:08.608238 30088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:08.608497 30088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:08.625032 30088 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:43369
I20260812 07:15:08.625068 30214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:43369 every 8 connection(s)
I20260812 07:15:08.626241 30088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:08.629700 30215 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:08.634719 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30088
I20260812 07:15:08.634878 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:08.635294 22229 external_mini_cluster.cc:1428] Setting key 6fc41af472bee0fe28618d1219ae8628
I20260812 07:15:08.640924 30215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd: Bootstrap starting.
I20260812 07:15:08.644423 30215 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:08.645350 30215 log.cc:824] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:08.648002 30215 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd: No bootstrap required, opened a new log
Aug 12 07:15:08 dist-test-slave-lgr5 krb5kdc[30058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:08.657581 30215 raft_consensus.cc:348] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515faed6f30b4649b5a2fa78d32da2bd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43369 } }
I20260812 07:15:08.658016 30215 raft_consensus.cc:374] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:08.658123 30215 raft_consensus.cc:729] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 515faed6f30b4649b5a2fa78d32da2bd, State: Initialized, Role: FOLLOWER
I20260812 07:15:08.658552 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:08.636899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54755 (local address 127.21.181.126:43369)
0812 07:15:08.637727 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:08.637741 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:08.637794 (+ 53us) server_negotiation.cc:407] Connection header received
0812 07:15:08.640062 (+ 2268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:08.640114 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:08.640577 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:08.641001 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:08.642304 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:08.644249 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:08.645445 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:08.646317 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:08.646500 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:08.650544 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:08.650554 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:08.650572 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:08.650612 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:08.654454 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:08.655079 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:08.655092 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:08.655099 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:08.655307 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:08.655634 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:08.655639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:08.655643 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:08.656342 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:08.656628 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:08.656928 (+ 300us) server_negotiation.cc:299] Negotiation successful
0812 07:15:08.657321 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":339,"threads_started":1}
I20260812 07:15:08.658766 30215 consensus_queue.cc:260] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515faed6f30b4649b5a2fa78d32da2bd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43369 } }
I20260812 07:15:08.659056 30215 raft_consensus.cc:388] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:08.659205 30215 raft_consensus.cc:482] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:08.659381 30215 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:08.662063 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:08.662004 30215 raft_consensus.cc:504] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515faed6f30b4649b5a2fa78d32da2bd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43369 } }
I20260812 07:15:08.662432 30215 leader_election.cc:302] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 515faed6f30b4649b5a2fa78d32da2bd; no voters:
I20260812 07:15:08.662977 30215 leader_election.cc:288] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:08.663102 30223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:08.663663 30223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [term 1 LEADER]: Becoming Leader. State: Replica: 515faed6f30b4649b5a2fa78d32da2bd, State: Running, Role: LEADER
I20260812 07:15:08.664284 30223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515faed6f30b4649b5a2fa78d32da2bd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43369 } }
I20260812 07:15:08.668454 30215 sys_catalog.cc:564] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:08.673957 30228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 515faed6f30b4649b5a2fa78d32da2bd. Latest consensus state: current_term: 1 leader_uuid: "515faed6f30b4649b5a2fa78d32da2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515faed6f30b4649b5a2fa78d32da2bd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43369 } } }
I20260812 07:15:08.674332 30228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:08.674881 30233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "515faed6f30b4649b5a2fa78d32da2bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "515faed6f30b4649b5a2fa78d32da2bd" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43369 } } }
I20260812 07:15:08.675181 30233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:08.678795 30270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:08.683431 30219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:08.667667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39129 (local address 127.21.181.126:43369)
0812 07:15:08.667815 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:08.667820 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:08.667839 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:15:08.668038 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:08.668045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:08.668154 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:08.668226 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:08.670277 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:08.671480 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:08.673229 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:08.673877 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:08.673973 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:08.677098 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:08.677104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:08.677110 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:08.677147 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:08.680915 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:08.681522 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:08.681528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:08.681535 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:08.681657 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:08.682002 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:08.682008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:08.682011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:08.682335 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:08.682593 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:08.682889 (+ 296us) server_negotiation.cc:299] Negotiation successful
0812 07:15:08.683099 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:15:08.684317 30270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:08.692821 30270 catalog_manager.cc:1347] Generated new cluster ID: 51022a6a44ce4c47adbc57b67e030c58
I20260812 07:15:08.692917 30270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:08.707687 30270 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:08.709340 30270 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:08.715467 30270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 515faed6f30b4649b5a2fa78d32da2bd: Generated new TSK 0
I20260812 07:15:08.716387 30270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:08.725468 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30088
I20260812 07:15:08.726468 30208 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:08.812764 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30088
2026-08-12T07:15:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2132 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:08 dist-test-slave-lgr5 krb5kdc[30281](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:08 dist-test-slave-lgr5 krb5kdc[30281](info): set up 2 sockets
Aug 12 07:15:08 dist-test-slave-lgr5 krb5kdc[30281](info): commencing operation
krb5kdc: starting...
W20260812 07:15:10.099598 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:10 dist-test-slave-lgr5 krb5kdc[30281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:10.375425 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43061
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:40295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:43061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:10.691300 30302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:10.691556 30302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:10.691656 30302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:10.698539 30302 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:10.698629 30302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:10.698681 30302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:10.698745 30302 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:10.698788 30302 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:10.707942 30302 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:40295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:43061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:10.709031 30302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:10.710198 30302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:10.720830 30314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:10.720904 30307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:10.720878 30308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:10.721611 30302 server_base.cc:1034] running on GCE node
I20260812 07:15:10.722278 30302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:10.723512 30302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:10.724702 30302 hybrid_clock.cc:648] HybridClock initialized: now 1786518910724659 us; error 37 us; skew 500 ppm
I20260812 07:15:10.725013 30302 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:10 dist-test-slave-lgr5 krb5kdc[30281](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518910, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:10.727890 30302 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:10.729208 30302 webserver.cc:460] Webserver started at http://127.21.181.126:32907/ using document root <none> and password file <none>
I20260812 07:15:10.729804 30302 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:10.729899 30302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:10.730137 30302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:10.732890 30302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "f18523110efd491e8317c78d3f793804"
format_stamp: "Formatted at 2026-08-12 07:15:10 on dist-test-slave-lgr5"
server_key: "6df75e2984a1214bdec9be607595322e"
server_key_iv: "5bc99eeae9c3a2f41d3064d709d4caf4"
server_key_version: "encryptionkey@0"
I20260812 07:15:10.733568 30302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "f18523110efd491e8317c78d3f793804"
format_stamp: "Formatted at 2026-08-12 07:15:10 on dist-test-slave-lgr5"
server_key: "6df75e2984a1214bdec9be607595322e"
server_key_iv: "5bc99eeae9c3a2f41d3064d709d4caf4"
server_key_version: "encryptionkey@0"
I20260812 07:15:10.738205 30302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260812 07:15:10.741348 30325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:10.742503 30302 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:15:10.742637 30302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "f18523110efd491e8317c78d3f793804"
format_stamp: "Formatted at 2026-08-12 07:15:10 on dist-test-slave-lgr5"
server_key: "6df75e2984a1214bdec9be607595322e"
server_key_iv: "5bc99eeae9c3a2f41d3064d709d4caf4"
server_key_version: "encryptionkey@0"
I20260812 07:15:10.742786 30302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:10.759125 30302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:10.763015 30302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:10.763232 30302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:10.779840 30302 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:43061
I20260812 07:15:10.779871 30408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:43061 every 8 connection(s)
I20260812 07:15:10.781059 30302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:10.784677 30410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:10.787792 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30302
I20260812 07:15:10.787967 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:10.788544 22229 external_mini_cluster.cc:1428] Setting key 47dd7403ae8b0b61f4e3944a5fbf1804
I20260812 07:15:10.796190 30410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804: Bootstrap starting.
I20260812 07:15:10.799403 30410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:10.800523 30410 log.cc:824] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:10.800822 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:10.790199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42717 (local address 127.21.181.126:43061)
0812 07:15:10.790910 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:10.790919 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:10.790957 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:15:10.792599 (+ 1642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:10.792620 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:10.792974 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:10.793270 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:10.793810 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:10.795098 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:10.796850 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:10.797477 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:10.797618 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:10.798752 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:10.798759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:10.798770 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:10.799237 (+ 467us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:10.799304 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:10.799332 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:10.799418 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:10.799660 (+ 242us) server_negotiation.cc:299] Negotiation successful
0812 07:15:10.799893 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":341,"threads_started":1}
I20260812 07:15:10.803009 30410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804: No bootstrap required, opened a new log
I20260812 07:15:10.803961 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:10.812390 30410 raft_consensus.cc:348] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f18523110efd491e8317c78d3f793804" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43061 } }
I20260812 07:15:10.812712 30410 raft_consensus.cc:374] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:10.812817 30410 raft_consensus.cc:729] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f18523110efd491e8317c78d3f793804, State: Initialized, Role: FOLLOWER
I20260812 07:15:10.813367 30410 consensus_queue.cc:260] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f18523110efd491e8317c78d3f793804" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43061 } }
I20260812 07:15:10.813606 30410 raft_consensus.cc:388] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:10.813726 30410 raft_consensus.cc:482] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:10.813868 30410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:10.814846 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:10.808895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39503 (local address 127.21.181.126:43061)
0812 07:15:10.809055 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:10.809061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:10.809599 (+ 538us) server_negotiation.cc:407] Connection header received
0812 07:15:10.809757 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:10.809761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:10.809818 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:10.809888 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:10.810389 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:10.811064 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:10.812644 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:10.813156 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:10.813258 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:10.814193 (+ 935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:10.814199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:10.814203 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:10.814220 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:10.814253 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:10.814271 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:10.814349 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:10.814487 (+ 138us) server_negotiation.cc:299] Negotiation successful
0812 07:15:10.814580 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:15:10.816561 30410 raft_consensus.cc:504] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f18523110efd491e8317c78d3f793804" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43061 } }
I20260812 07:15:10.816985 30410 leader_election.cc:302] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f18523110efd491e8317c78d3f793804; no voters:
I20260812 07:15:10.817507 30410 leader_election.cc:288] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:10.817615 30424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:10.818254 30424 raft_consensus.cc:686] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [term 1 LEADER]: Becoming Leader. State: Replica: f18523110efd491e8317c78d3f793804, State: Running, Role: LEADER
I20260812 07:15:10.818835 30424 consensus_queue.cc:237] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f18523110efd491e8317c78d3f793804" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43061 } }
I20260812 07:15:10.819546 30410 sys_catalog.cc:564] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:10.821398 30429 sys_catalog.cc:455] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f18523110efd491e8317c78d3f793804. Latest consensus state: current_term: 1 leader_uuid: "f18523110efd491e8317c78d3f793804" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f18523110efd491e8317c78d3f793804" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43061 } } }
I20260812 07:15:10.821421 30425 sys_catalog.cc:455] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f18523110efd491e8317c78d3f793804" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f18523110efd491e8317c78d3f793804" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 43061 } } }
I20260812 07:15:10.821848 30429 sys_catalog.cc:458] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:10.821848 30425 sys_catalog.cc:458] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:10.823104 30437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:10.827950 30437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:10.835772 30437 catalog_manager.cc:1347] Generated new cluster ID: 8648fc03833e4cb7b8a3938d2f94c536
I20260812 07:15:10.835877 30437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:10.869613 30437 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:10.870853 30437 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:10.879458 30437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f18523110efd491e8317c78d3f793804: Generated new TSK 0
I20260812 07:15:10.880245 30437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:10.894075 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30302
W20260812 07:15:10.895079 30402 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42717
W20260812 07:15:10.895709 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30302: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:15:10.895850 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30302
2026-08-12T07:15:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2083 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:11 dist-test-slave-lgr5 krb5kdc[30469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:11 dist-test-slave-lgr5 krb5kdc[30469](info): set up 2 sockets
Aug 12 07:15:11 dist-test-slave-lgr5 krb5kdc[30469](info): commencing operation
krb5kdc: starting...
W20260812 07:15:12.153851 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:12 dist-test-slave-lgr5 krb5kdc[30469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:12.425405 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:41789
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:42879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:41789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:12.744658 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:12.744901 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:12.745030 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:12.751859 30494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:12.751955 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:12.752031 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:12.752132 30494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:12.752187 30494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:12.761516 30494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:42879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:41789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:41789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:12.762631 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:12.763845 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:12.774636 30509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:12.774727 30505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:12.774693 30506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:12.775434 30494 server_base.cc:1034] running on GCE node
I20260812 07:15:12.776171 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:12.777438 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:12.778648 30494 hybrid_clock.cc:648] HybridClock initialized: now 1786518912778600 us; error 48 us; skew 500 ppm
I20260812 07:15:12.778960 30494 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:12 dist-test-slave-lgr5 krb5kdc[30469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518912, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:12.781965 30494 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:12.783303 30494 webserver.cc:460] Webserver started at http://127.21.181.126:38421/ using document root <none> and password file <none>
I20260812 07:15:12.783905 30494 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:12.784027 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:12.784330 30494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:12.787086 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "2d84786751b04a14bbd7a583b18a257b"
format_stamp: "Formatted at 2026-08-12 07:15:12 on dist-test-slave-lgr5"
server_key: "192f97bf6667022d7b7a5f0b4698053b"
server_key_iv: "e66e61e9badbc9caa7f6d401c91614f8"
server_key_version: "encryptionkey@0"
I20260812 07:15:12.787815 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "2d84786751b04a14bbd7a583b18a257b"
format_stamp: "Formatted at 2026-08-12 07:15:12 on dist-test-slave-lgr5"
server_key: "192f97bf6667022d7b7a5f0b4698053b"
server_key_iv: "e66e61e9badbc9caa7f6d401c91614f8"
server_key_version: "encryptionkey@0"
I20260812 07:15:12.792472 30494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260812 07:15:12.795624 30522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:12.796828 30494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:15:12.796983 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "2d84786751b04a14bbd7a583b18a257b"
format_stamp: "Formatted at 2026-08-12 07:15:12 on dist-test-slave-lgr5"
server_key: "192f97bf6667022d7b7a5f0b4698053b"
server_key_iv: "e66e61e9badbc9caa7f6d401c91614f8"
server_key_version: "encryptionkey@0"
I20260812 07:15:12.797156 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:12.802290 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:12.806058 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:12.806318 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:12.822495 30494 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:41789
I20260812 07:15:12.822528 30609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:41789 every 8 connection(s)
I20260812 07:15:12.823680 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:12.827054 30611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:12.829396 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30494
I20260812 07:15:12.829559 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:12.829954 22229 external_mini_cluster.cc:1428] Setting key 3305bd954c4d2807515075216cb22f11
I20260812 07:15:12.836905 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:12.831554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36019 (local address 127.21.181.126:41789)
0812 07:15:12.832300 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:12.832310 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:12.832348 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:15:12.833896 (+ 1548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:12.833917 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:12.834222 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:12.834528 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:12.835514 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:12.835520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:12.835531 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:12.835545 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:12.835604 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:12.835625 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:12.835791 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:12.835893 (+ 102us) server_negotiation.cc:299] Negotiation successful
0812 07:15:12.836068 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":300,"threads_started":1}
I20260812 07:15:12.838092 30611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b: Bootstrap starting.
I20260812 07:15:12.840255 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:12.841073 30611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:12.841982 30611 log.cc:824] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:12.844862 30611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b: No bootstrap required, opened a new log
I20260812 07:15:12.846501 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:12.844220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45255 (local address 127.21.181.126:41789)
0812 07:15:12.844375 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:12.844381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:12.844879 (+ 498us) server_negotiation.cc:407] Connection header received
0812 07:15:12.845045 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:12.845048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:12.845089 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:12.845158 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:12.845953 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:12.845956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:12.845960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0812 07:15:12.845984 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:12.846021 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0812 07:15:12.846035 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:12.846112 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:12.846235 (+ 123us) server_negotiation.cc:299] Negotiation successful
0812 07:15:12.846271 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:15:12.852054 30611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d84786751b04a14bbd7a583b18a257b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41789 } }
I20260812 07:15:12.852418 30611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 07:15:12.852526 30611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d84786751b04a14bbd7a583b18a257b, State: Initialized, Role: FOLLOWER
I20260812 07:15:12.853025 30611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d84786751b04a14bbd7a583b18a257b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41789 } }
I20260812 07:15:12.853236 30611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 07:15:12.853359 30611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 07:15:12.853509 30611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:12.855511 30611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d84786751b04a14bbd7a583b18a257b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41789 } }
I20260812 07:15:12.855939 30611 leader_election.cc:302] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d84786751b04a14bbd7a583b18a257b; no voters:
I20260812 07:15:12.856487 30611 leader_election.cc:288] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260812 07:15:12.856606 30634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:12.857182 30634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [term 1 LEADER]: Becoming Leader. State: Replica: 2d84786751b04a14bbd7a583b18a257b, State: Running, Role: LEADER
I20260812 07:15:12.857678 30634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d84786751b04a14bbd7a583b18a257b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41789 } }
I20260812 07:15:12.858270 30611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:12.860353 30637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d84786751b04a14bbd7a583b18a257b. Latest consensus state: current_term: 1 leader_uuid: "2d84786751b04a14bbd7a583b18a257b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d84786751b04a14bbd7a583b18a257b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41789 } } }
I20260812 07:15:12.860749 30637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:12.861377 30635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d84786751b04a14bbd7a583b18a257b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d84786751b04a14bbd7a583b18a257b" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 41789 } } }
I20260812 07:15:12.861711 30635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:12.861621 30647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:12.867126 30647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:12.878355 30647 catalog_manager.cc:1347] Generated new cluster ID: 149ab39a1c1a47b4b5c7b23449039c00
I20260812 07:15:12.878458 30647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:12.890225 30647 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:12.890693 30647 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:12.900357 30647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d84786751b04a14bbd7a583b18a257b: Generated new TSK 0
I20260812 07:15:12.901270 30647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:12.906105 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30494
W20260812 07:15:12.907079 30605 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36019
W20260812 07:15:12.907668 22229 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30494: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260812 07:15:12.907820 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30494
2026-08-12T07:15:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2008 ms)
[----------] 18 tests from AuthTokenIssuingTest (37419 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:13 dist-test-slave-lgr5 krb5kdc[30670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:13 dist-test-slave-lgr5 krb5kdc[30670](info): set up 2 sockets
Aug 12 07:15:13 dist-test-slave-lgr5 krb5kdc[30670](info): commencing operation
krb5kdc: starting...
W20260812 07:15:14.204933 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:14 dist-test-slave-lgr5 krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:14.475337 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39275
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:39275,127.21.181.125:43379,127.21.181.124:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:14.795671 30699 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:15:14.795982 30699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:14.796105 30699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:14.796231 30699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:14.803197 30699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:14.803282 30699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:14.803377 30699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:14.803442 30699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:14.803524 30699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:14.812728 30699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:39275,127.21.181.125:43379,127.21.181.124:39411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:39275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:14.813851 30699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:14.815093 30699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:14.825951 30713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:14.825985 30723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:14.825914 30716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:14.826968 30699 server_base.cc:1034] running on GCE node
I20260812 07:15:14.827651 30699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:14.828900 30699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:14.830111 30699 hybrid_clock.cc:648] HybridClock initialized: now 1786518914830057 us; error 30 us; skew 500 ppm
I20260812 07:15:14.830394 30699 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:14 dist-test-slave-lgr5 krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:14.836313 30699 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:14.837695 30699 webserver.cc:460] Webserver started at http://127.21.181.126:38129/ using document root <none> and password file <none>
I20260812 07:15:14.838289 30699 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:14.838379 30699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:14.838635 30699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:14.841465 30699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "464f174191c749a8a13ce39d9eb7beb2"
format_stamp: "Formatted at 2026-08-12 07:15:14 on dist-test-slave-lgr5"
server_key: "ed3a3c976468fa5eed94a68d72c975e7"
server_key_iv: "16815dc3d370d7ed4feef704866b1200"
server_key_version: "encryptionkey@0"
I20260812 07:15:14.842191 30699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "464f174191c749a8a13ce39d9eb7beb2"
format_stamp: "Formatted at 2026-08-12 07:15:14 on dist-test-slave-lgr5"
server_key: "ed3a3c976468fa5eed94a68d72c975e7"
server_key_iv: "16815dc3d370d7ed4feef704866b1200"
server_key_version: "encryptionkey@0"
I20260812 07:15:14.846843 30699 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:15:14.850035 30735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:14.851228 30699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260812 07:15:14.851364 30699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "464f174191c749a8a13ce39d9eb7beb2"
format_stamp: "Formatted at 2026-08-12 07:15:14 on dist-test-slave-lgr5"
server_key: "ed3a3c976468fa5eed94a68d72c975e7"
server_key_iv: "16815dc3d370d7ed4feef704866b1200"
server_key_version: "encryptionkey@0"
I20260812 07:15:14.851526 30699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:14.877540 30699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:14.881013 30699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:14.881229 30699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:14.898478 30699 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:39275
I20260812 07:15:14.898505 30815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:39275 every 8 connection(s)
I20260812 07:15:14.899741 30699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:14.903254 30818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:14.907927 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30699
I20260812 07:15:14.908169 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:14.909008 22229 external_mini_cluster.cc:1428] Setting key c71016bd4e42d074c7be8ca758e35fcd
I20260812 07:15:14.910619 30818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:14 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:14.933625 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:14.914330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39275 (local address 127.0.0.1:39556)
0812 07:15:14.915542 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:14.915593 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:14.917348 (+ 1755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:14.918283 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:14.918288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:14.918847 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:14.919185 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:14.919203 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:14.921210 (+ 2007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:14.921214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:14.922269 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:14.922281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:14.923429 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:14.923436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:14.923569 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:14.924217 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:14.924239 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:14.926420 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:14.929935 (+ 3515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:14.929945 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:14.929967 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:14.930352 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:14.930778 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:14.930781 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:14.930783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:14.930931 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:14.931746 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:14.931751 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:14.931954 (+ 203us) client_negotiation.cc:769] Sending connection context
0812 07:15:14.932195 (+ 241us) client_negotiation.cc:240] Negotiation successful
0812 07:15:14.932570 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":951,"spinlock_wait_cycles":44800,"thread_start_us":719,"threads_started":1}
I20260812 07:15:14.933665 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:14.914335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39556 (local address 127.21.181.126:39275)
0812 07:15:14.915175 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:14.915188 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:14.915723 (+ 535us) server_negotiation.cc:407] Connection header received
0812 07:15:14.917706 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:14.917713 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:14.918002 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:14.918209 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:14.919407 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:14.921060 (+ 1653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:14.922432 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:14.923191 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:14.923349 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:14.926647 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:14.926657 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:14.926672 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:14.926706 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:14.929805 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:14.930490 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:14.930497 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:14.930500 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:14.930675 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:14.931031 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:14.931035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:14.931037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:14.931642 (+ 605us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:14.931843 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:14.932360 (+ 517us) server_negotiation.cc:299] Negotiation successful
0812 07:15:14.932734 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":599,"thread_start_us":361,"threads_started":1}
I20260812 07:15:14.936474 30818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:14.937433 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:14.937075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56158)
0812 07:15:14.937232 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:14.937336 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260812 07:15:14.937682 30737 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
I20260812 07:15:14.939124 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:14.938796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56168)
0812 07:15:14.938941 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:14.939014 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260812 07:15:14.939322 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:43379: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.21.181.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab.
Entry for principal kudu/127.21.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab.
I20260812 07:15:14.980340 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:43379
--webserver_interface=127.21.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab
--principal=kudu/127.21.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:39275,127.21.181.125:43379,127.21.181.124:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:15:14.988641 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } attempt: 1
I20260812 07:15:14.989894 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:14.989346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56178)
0812 07:15:14.989593 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:14.989731 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260812 07:15:14.990878 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:14.990484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56182)
0812 07:15:14.990642 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:14.990723 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:15:14.991066 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:43379: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
I20260812 07:15:15.059281 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } attempt: 2
I20260812 07:15:15.060494 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.060001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56188)
0812 07:15:15.060256 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.060348 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260812 07:15:15.061455 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.061116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56190)
0812 07:15:15.061257 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.061336 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260812 07:15:15.061636 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:43379: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
I20260812 07:15:15.152877 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } attempt: 3
I20260812 07:15:15.153925 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.153496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56196)
0812 07:15:15.153697 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.153788 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:15:15.154887 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.154544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56212)
0812 07:15:15.154687 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.154753 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:15:15.155140 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:43379: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
I20260812 07:15:15.298373 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } attempt: 4
I20260812 07:15:15.299566 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.299018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56228)
0812 07:15:15.299331 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.299418 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260812 07:15:15.300695 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.300310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56230)
0812 07:15:15.300492 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.300576 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260812 07:15:15.300902 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:43379: Network error: Client connection negotiation failed: client connection to 127.21.181.125:43379: connect: Connection refused (error 111)
W20260812 07:15:15.310223 30833 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:15:15.310523 30833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:15.310612 30833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:15.310724 30833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:15.317767 30833 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:15.317842 30833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:15.317929 30833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:15.317988 30833 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:15.318079 30833 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:15.327281 30833 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:39275,127.21.181.125:43379,127.21.181.124:39411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab
--principal=kudu/127.21.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:43379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:15.328392 30833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:15.329615 30833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:15.340260 30843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:15.340288 30847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:15.340554 30851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:15.341285 30833 server_base.cc:1034] running on GCE node
I20260812 07:15:15.341988 30833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:15.343184 30833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:15.344393 30833 hybrid_clock.cc:648] HybridClock initialized: now 1786518915344323 us; error 38 us; skew 500 ppm
I20260812 07:15:15.344671 30833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:15.347472 30833 init.cc:377] Logged in from keytab as kudu/127.21.181.125@KRBTEST.COM (short username kudu)
I20260812 07:15:15.348768 30833 webserver.cc:460] Webserver started at http://127.21.181.125:36249/ using document root <none> and password file <none>
I20260812 07:15:15.349391 30833 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:15.349494 30833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:15.349784 30833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:15.352592 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data/instance:
uuid: "489cbbd325d64e71ab6d4d75567fba4b"
format_stamp: "Formatted at 2026-08-12 07:15:15 on dist-test-slave-lgr5"
server_key: "cc8d7d654ba62ba2a78cf05fc9aae9a1"
server_key_iv: "c190672ad7a6f60ba10dafaeab7fb71a"
server_key_version: "encryptionkey@0"
I20260812 07:15:15.353332 30833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal/instance:
uuid: "489cbbd325d64e71ab6d4d75567fba4b"
format_stamp: "Formatted at 2026-08-12 07:15:15 on dist-test-slave-lgr5"
server_key: "cc8d7d654ba62ba2a78cf05fc9aae9a1"
server_key_iv: "c190672ad7a6f60ba10dafaeab7fb71a"
server_key_version: "encryptionkey@0"
I20260812 07:15:15.358059 30833 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260812 07:15:15.361215 30867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:15.362416 30833 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260812 07:15:15.362571 30833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal
uuid: "489cbbd325d64e71ab6d4d75567fba4b"
format_stamp: "Formatted at 2026-08-12 07:15:15 on dist-test-slave-lgr5"
server_key: "cc8d7d654ba62ba2a78cf05fc9aae9a1"
server_key_iv: "c190672ad7a6f60ba10dafaeab7fb71a"
server_key_version: "encryptionkey@0"
I20260812 07:15:15.362753 30833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:15.386158 30833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:15.389675 30833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:15.389920 30833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:15.406316 30833 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.125:43379
I20260812 07:15:15.406335 30954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.125:43379 every 8 connection(s)
I20260812 07:15:15.407488 30833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
I20260812 07:15:15.409078 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30833
I20260812 07:15:15.409232 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-1/wal/instance
I20260812 07:15:15.409698 22229 external_mini_cluster.cc:1428] Setting key e6a7574f618c01888da6da75e380c38b
I20260812 07:15:15.410992 30955 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:15.419512 30955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:15.442389 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.421958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39560 (local address 127.21.181.126:39275)
0812 07:15:15.422191 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.422197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.423362 (+ 1165us) server_negotiation.cc:407] Connection header received
0812 07:15:15.425338 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.425345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.425438 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.425556 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.427431 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.428599 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.430644 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.431350 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.431477 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.435704 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.435712 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.435716 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.435756 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.439152 (+ 3396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.440098 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.440105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.440108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.440261 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.440750 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.440756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.440760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.441061 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.441210 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.441852 (+ 642us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.442071 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 07:15:15.443302 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.422491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39275 (local address 127.0.0.1:39560)
0812 07:15:15.423206 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.423259 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:15.424991 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:15.425813 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:15.425823 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:15.426517 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:15.427209 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.427232 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.428774 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.428779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.430476 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.430492 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.431586 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.431593 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.431852 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.432629 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:15.432655 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:15.435450 (+ 2795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:15.439302 (+ 3852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.439311 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.439335 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.439923 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.440391 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.440396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.440400 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.440627 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.441201 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:15.441208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:15.441505 (+ 297us) client_negotiation.cc:769] Sending connection context
0812 07:15:15.441733 (+ 228us) client_negotiation.cc:240] Negotiation successful
0812 07:15:15.442221 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":285,"threads_started":1}
WARNING: no policy specified for kudu/127.21.181.124@KRBTEST.COM; defaulting to no policy
I20260812 07:15:15.444913 30955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.124@KRBTEST.COM" created.
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:15:15.462597 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.445595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56240)
0812 07:15:15.445735 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.445752 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:15.445959 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:15.446984 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:15.446988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:15.447367 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:15.447517 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.447526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.449571 (+ 2045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.449575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.450716 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.450731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.452194 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.452200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.452350 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.452953 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:15.452973 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:15.455488 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:15.459622 (+ 4134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.459627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.459631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.460152 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.460546 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.460552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.460556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.460747 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.461482 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:15.461488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:15.461659 (+ 171us) client_negotiation.cc:769] Sending connection context
0812 07:15:15.461834 (+ 175us) client_negotiation.cc:240] Negotiation successful
0812 07:15:15.462130 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260812 07:15:15.462671 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.445843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56240 (local address 127.21.181.125:43379)
0812 07:15:15.446503 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.446512 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.446545 (+ 33us) server_negotiation.cc:407] Connection header received
0812 07:15:15.446601 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.446609 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.446819 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.446976 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.447765 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.449416 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.450872 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.451873 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.452022 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.455745 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.455761 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.455777 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.455817 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.459450 (+ 3633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.460285 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.460294 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.460298 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.460435 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.460877 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.460881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.460884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.461328 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.461568 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.461984 (+ 416us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.462274 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":481,"thread_start_us":332,"threads_started":1}
I20260812 07:15:15.465597 30955 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:15.466747 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.466257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40474)
0812 07:15:15.466427 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.466575 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260812 07:15:15.467061 30871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
I20260812 07:15:15.468711 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.468314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40486)
0812 07:15:15.468508 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.468598 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260812 07:15:15.469007 30955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab.
Entry for principal kudu/127.21.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab.
I20260812 07:15:15.481418 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:39411
--webserver_interface=127.21.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab
--principal=kudu/127.21.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:35417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:39275,127.21.181.125:43379,127.21.181.124:39411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:15:15.518321 30955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 1
I20260812 07:15:15.519629 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.519097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40498)
0812 07:15:15.519316 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.519441 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260812 07:15:15.520773 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.520368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40512)
0812 07:15:15.520542 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.520638 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260812 07:15:15.521073 30955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
I20260812 07:15:15.589301 30955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 2
I20260812 07:15:15.590590 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.590020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40514)
0812 07:15:15.590259 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.590376 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260812 07:15:15.591795 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.591362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40518)
0812 07:15:15.591535 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.591634 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260812 07:15:15.592154 30955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
I20260812 07:15:15.601191 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } attempt: 5
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:15:15.617686 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.601887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56244 (local address 127.21.181.125:43379)
0812 07:15:15.602055 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.602061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.602169 (+ 108us) server_negotiation.cc:407] Connection header received
0812 07:15:15.602372 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.602378 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.602457 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.602571 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.603518 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.604669 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.606459 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.607097 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.607227 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.611251 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.611259 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.611264 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.611306 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.614766 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.615609 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.615615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.615619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.615762 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.616209 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.616214 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.616218 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.616582 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.616729 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.617154 (+ 425us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.617375 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:15:15.617710 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.601816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56244)
0812 07:15:15.602078 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.602095 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:15.602238 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:15.602613 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:15.602618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:15.603118 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:15.603355 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.603369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.604842 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.604849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.606286 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.606304 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.607256 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.607264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.607533 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.608222 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:15.608244 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:15.611045 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:15.614912 (+ 3867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.614918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.614922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.615481 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.615885 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.615889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.615893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.616102 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.616711 (+ 609us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:15.616715 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:15.616878 (+ 163us) client_negotiation.cc:769] Sending connection context
0812 07:15:15.617036 (+ 158us) client_negotiation.cc:240] Negotiation successful
0812 07:15:15.617333 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143}
I20260812 07:15:15.618860 30818 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:15.619958 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.619521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40526)
0812 07:15:15.619704 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.619797 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:15:15.620983 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.620611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40534)
0812 07:15:15.620769 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.620853 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:15:15.621243 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
I20260812 07:15:15.672451 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 1
I20260812 07:15:15.673520 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.673072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40542)
0812 07:15:15.673264 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.673351 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:15:15.674427 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.674117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40558)
0812 07:15:15.674258 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.674325 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:15:15.674702 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
I20260812 07:15:15.683416 30955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 3
I20260812 07:15:15.684454 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.684000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40568)
0812 07:15:15.684205 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.684293 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260812 07:15:15.685544 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.685107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40572)
0812 07:15:15.685337 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.685427 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260812 07:15:15.685794 30955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
I20260812 07:15:15.742930 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 2
I20260812 07:15:15.744374 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.743705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40586)
0812 07:15:15.743985 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.744147 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260812 07:15:15.745635 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.745220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40600)
0812 07:15:15.745388 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.745498 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260812 07:15:15.745895 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
W20260812 07:15:15.826526 30968 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:15:15.826807 30968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:15.826889 30968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:15.826965 30968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 07:15:15.828990 30955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 4
I20260812 07:15:15.830073 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.829600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40612)
0812 07:15:15.829816 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.829897 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260812 07:15:15.830943 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.830651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40614)
0812 07:15:15.830787 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.830851 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:15:15.831133 30955 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
W20260812 07:15:15.834034 30968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:15.834118 30968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:15.834209 30968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:15.834275 30968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:15.834358 30968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:15.843622 30968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:35417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:39275,127.21.181.125:43379,127.21.181.124:39411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab
--principal=kudu/127.21.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:39411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:15.844833 30968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:15.846051 30968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260812 07:15:15.852124 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 3
I20260812 07:15:15.853066 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.852679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40616)
0812 07:15:15.852844 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.852921 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:15:15.854017 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.853635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40626)
0812 07:15:15.853837 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.853909 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260812 07:15:15.854231 30818 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:39411: Network error: Client connection negotiation failed: client connection to 127.21.181.124:39411: connect: Connection refused (error 111)
W20260812 07:15:15.857434 30975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:15.857470 30976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:15.857388 30978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:15.858433 30968 server_base.cc:1034] running on GCE node
I20260812 07:15:15.859130 30968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:15.860369 30968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:15.861575 30968 hybrid_clock.cc:648] HybridClock initialized: now 1786518915861519 us; error 57 us; skew 500 ppm
I20260812 07:15:15.861845 30968 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:15.864725 30968 init.cc:377] Logged in from keytab as kudu/127.21.181.124@KRBTEST.COM (short username kudu)
I20260812 07:15:15.866055 30968 webserver.cc:460] Webserver started at http://127.21.181.124:44821/ using document root <none> and password file <none>
I20260812 07:15:15.866703 30968 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:15.866829 30968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:15.867126 30968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:15.869949 30968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data/instance:
uuid: "d41e6bbd3d614a639a07bdb739a7c8e3"
format_stamp: "Formatted at 2026-08-12 07:15:15 on dist-test-slave-lgr5"
server_key: "37b86284199542f5442f3b4357ca8911"
server_key_iv: "d0dcd26dfde8de411ff6e4daadced868"
server_key_version: "encryptionkey@0"
I20260812 07:15:15.870687 30968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal/instance:
uuid: "d41e6bbd3d614a639a07bdb739a7c8e3"
format_stamp: "Formatted at 2026-08-12 07:15:15 on dist-test-slave-lgr5"
server_key: "37b86284199542f5442f3b4357ca8911"
server_key_iv: "d0dcd26dfde8de411ff6e4daadced868"
server_key_version: "encryptionkey@0"
I20260812 07:15:15.875491 30968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:15:15.878696 30990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:15.879871 30968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:15:15.880028 30968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal
uuid: "d41e6bbd3d614a639a07bdb739a7c8e3"
format_stamp: "Formatted at 2026-08-12 07:15:15 on dist-test-slave-lgr5"
server_key: "37b86284199542f5442f3b4357ca8911"
server_key_iv: "d0dcd26dfde8de411ff6e4daadced868"
server_key_version: "encryptionkey@0"
I20260812 07:15:15.880254 30968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:15.898559 30968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:15.902568 30968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:15.902807 30968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:15.921222 30968 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.124:39411
I20260812 07:15:15.921255 31095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.124:39411 every 8 connection(s)
I20260812 07:15:15.922542 30968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
I20260812 07:15:15.926019 31096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:15.928748 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 30968
I20260812 07:15:15.928942 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786518562800324-22229-0/minicluster-data/master-2/wal/instance
I20260812 07:15:15.929409 22229 external_mini_cluster.cc:1428] Setting key 1d9248ae33bf68df6e0511697de0a33b
I20260812 07:15:15.933871 31096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:15.952200 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.936960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39570 (local address 127.21.181.126:39275)
0812 07:15:15.937186 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.937194 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.937793 (+ 599us) server_negotiation.cc:407] Connection header received
0812 07:15:15.937966 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.937970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.938037 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.938123 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.939411 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.940534 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.942357 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.943010 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.943157 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.946535 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.946541 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.946544 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.946579 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.949235 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.949967 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.949971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.949974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.950163 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.950579 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.950583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.950585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.950864 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.951025 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.951614 (+ 589us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.951842 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:15:15.953083 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.936939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:39275 (local address 127.0.0.1:39570)
0812 07:15:15.937641 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.937682 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:15.937843 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:15.938202 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:15.938209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:15.938998 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:15.939258 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.939267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.940697 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.940702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.942195 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.942205 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.943220 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.943225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.943344 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.943927 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:15.943954 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:15.946317 (+ 2363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:15.949365 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.949374 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.949381 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.949842 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.950272 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.950277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.950280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.950475 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.951000 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:15.951007 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:15.951227 (+ 220us) client_negotiation.cc:769] Sending connection context
0812 07:15:15.951461 (+ 234us) client_negotiation.cc:240] Negotiation successful
0812 07:15:15.951940 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":124,"thread_start_us":279,"threads_started":1}
I20260812 07:15:15.953104 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.931268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40628 (local address 127.21.181.124:39411)
0812 07:15:15.932121 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.932138 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.932186 (+ 48us) server_negotiation.cc:407] Connection header received
0812 07:15:15.934308 (+ 2122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.934337 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.934826 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.935248 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.936998 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.938909 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.940688 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.941740 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.941900 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.946195 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.946205 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.946219 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.946256 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.948976 (+ 2720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.949747 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.949756 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.949759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.949952 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.950324 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.950328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.950331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.950842 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.951163 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.951450 (+ 287us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.951796 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":574,"spinlock_wait_cycles":14592,"thread_start_us":347,"threads_started":1}
I20260812 07:15:15.954869 31096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:15.967239 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.955700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:43379 (local address 127.0.0.1:56260)
0812 07:15:15.955850 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.955869 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:15.956003 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:15.956386 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:15.956391 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:15.956790 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:15.956927 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.956937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.957839 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.957844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.958682 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.958694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.959374 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.959381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.959581 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.960027 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:15.960043 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:15.961914 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:15.964928 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.964933 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.964937 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.965277 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.965652 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.965657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.965661 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.965872 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.966339 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:15.966343 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:15.966484 (+ 141us) client_negotiation.cc:769] Sending connection context
0812 07:15:15.966612 (+ 128us) client_negotiation.cc:240] Negotiation successful
0812 07:15:15.966891 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260812 07:15:15.967303 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.955648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.21.181.125:43379)
0812 07:15:15.955854 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.955860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.955959 (+ 99us) server_negotiation.cc:407] Connection header received
0812 07:15:15.956180 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.956183 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.956244 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.956330 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.957063 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.957711 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.958812 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.959235 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.959343 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.962118 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.962124 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.962127 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.962163 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.964806 (+ 2643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.965387 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.965393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.965397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.965545 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.965971 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.965976 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.965979 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.966224 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.966352 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.966715 (+ 363us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.966941 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:15:15.968176 31096 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:15.970201 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.956724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39576 (local address 127.21.181.126:39275)
0812 07:15:15.956900 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.956906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.956924 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:15:15.956974 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.956979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.957038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.957124 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.958272 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.958983 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.960287 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.960726 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.960846 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.964804 (+ 3958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.964810 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.964813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.964841 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.967366 (+ 2525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.968028 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.968033 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.968037 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.968214 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.968562 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.968567 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.968571 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.968831 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.969094 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.969711 (+ 617us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.969903 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
Aug 12 07:15:15 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:15:15.982429 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.968748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40644)
0812 07:15:15.968876 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:15.968892 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:15.968978 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:15.969279 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:15.969282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:15.969596 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:15.969716 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.969723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.970798 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.970803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.972004 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:15.972022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.973039 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.973045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:15.973277 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.973839 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:15.973860 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:15.976181 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:15.980038 (+ 3857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.980043 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.980047 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.980559 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.980942 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:15.980945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:15.980947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:15.981077 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:15.981685 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:15.981689 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:15.981821 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 07:15:15.981931 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 07:15:15.982140 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260812 07:15:15.982643 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.968921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.21.181.124:39411)
0812 07:15:15.969067 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.969072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.969087 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:15:15.969131 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.969136 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.969186 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.969256 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.969842 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.970684 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.972225 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.972879 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.972995 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.976376 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.976382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.976386 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.976420 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.979862 (+ 3442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.980675 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.980679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.980681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.980824 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.981193 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.981198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.981202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.981575 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.981721 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.982080 (+ 359us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.982323 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260812 07:15:15.988032 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:15.971874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56270 (local address 127.21.181.125:43379)
0812 07:15:15.972125 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:15.972130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:15.972145 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:15:15.972189 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:15.972194 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:15.972255 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:15.972347 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:15.973877 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.975245 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.976995 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:15.977434 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:15.977551 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:15.982445 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:15.982452 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:15.982456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:15.982623 (+ 167us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:15.985759 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.986326 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.986332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.986336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.986485 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:15.986792 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:15.986797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:15.986799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:15.987114 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:15.987376 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:15.987517 (+ 141us) server_negotiation.cc:299] Negotiation successful
0812 07:15:15.987725 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260812 07:15:15.988483 31096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: Bootstrap starting.
I20260812 07:15:15.991878 31096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:15.992599 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:15.992945 31096 log.cc:824] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:15.995690 31096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: No bootstrap required, opened a new log
I20260812 07:15:15.995132 30778 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "464f174191c749a8a13ce39d9eb7beb2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39576
I20260812 07:15:16.006099 31096 raft_consensus.cc:348] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.006502 31096 raft_consensus.cc:729] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41e6bbd3d614a639a07bdb739a7c8e3, State: Initialized, Role: FOLLOWER
I20260812 07:15:16.006989 31096 consensus_queue.cc:260] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.007846 31106 sys_catalog.cc:455] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.008212 31106 sys_catalog.cc:458] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.008911 31096 sys_catalog.cc:564] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:15:16.017808 31121 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:15:16.017920 31121 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:15:16.015501 30968 master_runner.cc:186] Error getting master registration for 127.21.181.126:39275: OK, instance_id { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" instance_seqno: 1786518914893537 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:15:16.023468 30968 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:15:16.031471 30818 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 4
Aug 12 07:15:16 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518914, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:15:16.047729 30826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.032000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40650)
0812 07:15:16.032232 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:16.032253 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:16.032384 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:16.032739 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:16.032744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:16.033278 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:16.033501 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:16.033515 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.034930 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.034936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:16.036416 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:16.036434 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.037448 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.037456 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:16.037671 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.038321 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:16.038346 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:16.041118 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:16.044901 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:16.044907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:16.044910 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:16.045385 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:16.045838 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:16.045843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:16.045847 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:16.046008 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:16.046788 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:16.046793 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:16.046957 (+ 164us) client_negotiation.cc:769] Sending connection context
0812 07:15:16.047096 (+ 139us) client_negotiation.cc:240] Negotiation successful
0812 07:15:16.047353 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:15:16.048647 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.032059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.21.181.124:39411)
0812 07:15:16.032261 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.032265 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.032331 (+ 66us) server_negotiation.cc:407] Connection header received
0812 07:15:16.032499 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.032505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.032583 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.032685 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.033643 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.034774 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.036557 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.037281 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.037396 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.041280 (+ 3884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:16.041285 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:16.041288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:16.041316 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:16.044739 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.045541 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.045547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.045550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.045705 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.046139 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.046144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.046148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.046579 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:16.046721 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.048066 (+ 1345us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.048305 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260812 07:15:16.056263 30818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: Bootstrap starting.
I20260812 07:15:16.060120 30818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:16.061412 30818 log.cc:824] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:16.064577 30818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: No bootstrap required, opened a new log
I20260812 07:15:16.071803 30818 raft_consensus.cc:348] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.072187 30818 raft_consensus.cc:729] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464f174191c749a8a13ce39d9eb7beb2, State: Initialized, Role: FOLLOWER
I20260812 07:15:16.072706 30818 consensus_queue.cc:260] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.073858 31129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.074476 31129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.074975 30818 sys_catalog.cc:564] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:16.075021 30778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:15:16.075199 30778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:16.078615 30778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.081189 31051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "464f174191c749a8a13ce39d9eb7beb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3"
I20260812 07:15:16.081779 31051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:16.085337 31051 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464f174191c749a8a13ce39d9eb7beb2 in term 1.
I20260812 07:15:16.085922 30738 leader_election.cc:302] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464f174191c749a8a13ce39d9eb7beb2, d41e6bbd3d614a639a07bdb739a7c8e3; no voters:
I20260812 07:15:16.086385 30778 leader_election.cc:288] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [CANDIDATE]: Term 1 election: Requested vote from peers 489cbbd325d64e71ab6d4d75567fba4b (127.21.181.125:43379), d41e6bbd3d614a639a07bdb739a7c8e3 (127.21.181.124:39411)
I20260812 07:15:16.093070 31129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:16.085428 30699 master_runner.cc:186] Error getting master registration for 127.21.181.125:43379: OK, instance_id { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" instance_seqno: 1786518915401667 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:15:16.096176 30699 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260812 07:15:16.102680 31150 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:15:16.102862 31150 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:15:16.103122 31129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [term 1 LEADER]: Becoming Leader. State: Replica: 464f174191c749a8a13ce39d9eb7beb2, State: Running, Role: LEADER
I20260812 07:15:16.103665 31129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.109282 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 464f174191c749a8a13ce39d9eb7beb2. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.109647 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:16.113065 31165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:16.116636 31165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260812 07:15:16.130331 30955 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } attempt: 5
W20260812 07:15:16.133734 30746 tablet.cc:2369] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:15:16.136878 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.105722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.21.181.125:43379)
0812 07:15:16.105910 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.105917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.120298 (+ 14381us) server_negotiation.cc:407] Connection header received
0812 07:15:16.120568 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.120574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.120652 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.120720 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.123023 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.124143 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.126093 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.126745 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.126838 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.129919 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:16.129926 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:16.129931 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:16.129991 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:16.133647 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.134523 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.134529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.134533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.134677 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.135138 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.135144 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.135148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.135496 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:16.135717 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.136302 (+ 585us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.136535 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:15:16.140045 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.105405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39590 (local address 127.21.181.126:39275)
0812 07:15:16.105577 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.105583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.115003 (+ 9420us) server_negotiation.cc:407] Connection header received
0812 07:15:16.115305 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.115314 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.115394 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.115484 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.117768 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.118918 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.120823 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.121441 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.121532 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.125087 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:16.125095 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:16.125100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:16.125149 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:16.128665 (+ 3516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.129517 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.129523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.129526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.129705 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.130163 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.130169 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.130172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.130512 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:16.131970 (+ 1458us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.139477 (+ 7507us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.139704 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:15:16.150811 31051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [term 1 FOLLOWER]: Refusing update from remote peer 464f174191c749a8a13ce39d9eb7beb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:15:16.151950 31153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 07:15:16.155043 31000 tablet.cc:2369] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:15:16.156397 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.121714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.21.181.124:39411)
0812 07:15:16.121887 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.121893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.139341 (+ 17448us) server_negotiation.cc:407] Connection header received
0812 07:15:16.139591 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.139597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.139692 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.139761 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.141439 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.142512 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.144331 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.145048 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.145147 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.148267 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:16.148274 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:16.148279 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:16.148316 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:16.151925 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.152854 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.152864 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.152868 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.153040 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.153483 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.153489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.153493 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.153822 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:16.154089 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.154588 (+ 499us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.155964 (+ 1376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 12 07:15:16 dist-test-slave-lgr5 krb5kdc[30670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518915, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:15:16.175172 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.130857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:39411 (local address 127.0.0.1:40668)
0812 07:15:16.131192 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:16.131210 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:16.131303 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:16.158064 (+ 26761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:16.158069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:16.158561 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:16.158777 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:16.158790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.160672 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.160678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:16.162044 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:16.162061 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.163067 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.163074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:16.163328 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.163921 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:16.163943 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:16.166877 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:16.171545 (+ 4668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:16.171550 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:16.171554 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:16.172050 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:16.172874 (+ 824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:16.172880 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:16.172884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:16.173102 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:16.174194 (+ 1092us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:16.174199 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:16.174356 (+ 157us) client_negotiation.cc:769] Sending connection context
0812 07:15:16.174500 (+ 144us) client_negotiation.cc:240] Negotiation successful
0812 07:15:16.174777 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260812 07:15:16.177064 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.131496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.21.181.124:39411)
0812 07:15:16.147688 (+ 16192us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.147694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.147716 (+ 22us) server_negotiation.cc:407] Connection header received
0812 07:15:16.147763 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.147767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.147833 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.158945 (+ 11112us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.159240 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.160519 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.162233 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.162902 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.165499 (+ 2597us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.167537 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:16.167548 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:16.167553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:16.167595 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:16.171361 (+ 3766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.172386 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.172394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.172398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.172671 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:16.173294 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:16.173299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:16.173303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:16.173991 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:16.175690 (+ 1699us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.175917 (+ 227us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.176269 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16057,"thread_start_us":16395,"threads_started":1}
I20260812 07:15:16.177103 31106 sys_catalog.cc:455] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 464f174191c749a8a13ce39d9eb7beb2. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.177595 31106 sys_catalog.cc:458] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.182855 30955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: Bootstrap starting.
I20260812 07:15:16.187134 30955 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:16.187863 31106 sys_catalog.cc:455] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.188128 31106 sys_catalog.cc:458] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.188505 30955 log.cc:824] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:16.201964 30955 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: No bootstrap required, opened a new log
I20260812 07:15:16.205327 31153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.205607 31153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:16.209293 31165 catalog_manager.cc:1347] Generated new cluster ID: b7ab7a0f30c64de3af10b4c5837a46ab
I20260812 07:15:16.209403 31165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:16.227546 30955 raft_consensus.cc:348] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.231051 30955 raft_consensus.cc:729] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489cbbd325d64e71ab6d4d75567fba4b, State: Initialized, Role: FOLLOWER
I20260812 07:15:16.231631 30955 consensus_queue.cc:260] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } }
I20260812 07:15:16.234937 31165 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:16.236724 31165 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:16.245750 31184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.247891 31184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.247769 30955 sys_catalog.cc:564] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:16.248174 30916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:16.248827 30917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "464f174191c749a8a13ce39d9eb7beb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "489cbbd325d64e71ab6d4d75567fba4b"
I20260812 07:15:16.252913 30916 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [term 1 FOLLOWER]: Refusing update from remote peer 464f174191c749a8a13ce39d9eb7beb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:15:16.256558 30917 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464f174191c749a8a13ce39d9eb7beb2 in term 1.
I20260812 07:15:16.258594 31153 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:15:16.265746 31165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2: Generated new TSK 0
I20260812 07:15:16.266827 31165 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260812 07:15:16.318090 31193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.318564 31193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.318643 31184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 464f174191c749a8a13ce39d9eb7beb2. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.318957 31184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:16.320570 31129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "464f174191c749a8a13ce39d9eb7beb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "464f174191c749a8a13ce39d9eb7beb2" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 39275 } } peers { permanent_uuid: "489cbbd325d64e71ab6d4d75567fba4b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 43379 } } peers { permanent_uuid: "d41e6bbd3d614a639a07bdb739a7c8e3" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 39411 } } }
I20260812 07:15:16.320910 31129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464f174191c749a8a13ce39d9eb7beb2 [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:16.349645 31214 catalog_manager.cc:1259] Loaded cluster ID: b7ab7a0f30c64de3af10b4c5837a46ab
I20260812 07:15:16.349776 31214 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: loading cluster ID for follower catalog manager: success
I20260812 07:15:16.353930 31214 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: acquiring CA information for follower catalog manager: success
I20260812 07:15:16.356526 31214 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 489cbbd325d64e71ab6d4d75567fba4b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:15:16.380656 22229 mini_kdc.cc:348] Time spent kdestroy: real 0.103s user 0.003s sys 0.028s
W20260812 07:15:16.438103 30823 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.414035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39604 (local address 127.21.181.126:39275)
0812 07:15:16.414243 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.414249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.435999 (+ 21750us) server_negotiation.cc:407] Connection header received
0812 07:15:16.436545 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.436552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.436646 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.436721 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.437853 (+ 1132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39604: BlockingRecv error: recv got EOF from 127.0.0.1:39604 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:15:16.440714 30961 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.420594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.21.181.125:43379)
0812 07:15:16.420807 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.420813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.438886 (+ 18073us) server_negotiation.cc:407] Connection header received
0812 07:15:16.439360 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.439366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.439490 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.439561 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.440509 (+ 948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56274: BlockingRecv error: recv got EOF from 127.0.0.1:56274 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:15:16.444007 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.441994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.21.181.124:39411)
0812 07:15:16.442152 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.442158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.442605 (+ 447us) server_negotiation.cc:407] Connection header received
0812 07:15:16.442820 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.442825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.442913 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.442981 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.443803 (+ 822us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40678: BlockingRecv error: recv got EOF from 127.0.0.1:40678 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260812 07:15:16.496290 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.476999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.21.181.125:43379)
0812 07:15:16.477193 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.477200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.487922 (+ 10722us) server_negotiation.cc:407] Connection header received
0812 07:15:16.488531 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.488539 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.488812 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.488972 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.489752 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.490947 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.493142 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.493841 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.493943 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.494714 (+ 771us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.495394 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.495522 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.495768 (+ 246us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.495887 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 07:15:16.543020 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.529494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.21.181.124:39411)
0812 07:15:16.529689 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.529695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.540705 (+ 11010us) server_negotiation.cc:407] Connection header received
0812 07:15:16.541284 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.541292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.541429 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.541514 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.542692 (+ 1178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40690: BlockingRecv error: recv got EOF from 127.0.0.1:40690 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:15:16.598131 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.579343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56298 (local address 127.21.181.125:43379)
0812 07:15:16.579558 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.579565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.590226 (+ 10661us) server_negotiation.cc:407] Connection header received
0812 07:15:16.590812 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.590821 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.590930 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.591007 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.591765 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.593000 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.595069 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.595769 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.595901 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.597111 (+ 1210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.597300 (+ 189us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.597383 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.597649 (+ 266us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.597742 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260812 07:15:16.612694 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.610273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.21.181.124:39411)
0812 07:15:16.610474 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.610480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.611082 (+ 602us) server_negotiation.cc:407] Connection header received
0812 07:15:16.611259 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.611264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.611358 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.611472 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.612477 (+ 1005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40692: BlockingRecv error: recv got EOF from 127.0.0.1:40692 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:15:16.625090 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.617982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56312 (local address 127.21.181.125:43379)
0812 07:15:16.618317 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.618324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.618863 (+ 539us) server_negotiation.cc:407] Connection header received
0812 07:15:16.619046 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.619052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.619130 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.619225 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.619731 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.620956 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.622908 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.623512 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.623640 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.624260 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.624422 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.624508 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.624686 (+ 178us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.624761 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193}
W20260812 07:15:16.634239 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.631945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.21.181.124:39411)
0812 07:15:16.632165 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.632171 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.632721 (+ 550us) server_negotiation.cc:407] Connection header received
0812 07:15:16.632917 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.632923 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.633023 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.633129 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.634016 (+ 887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40698: BlockingRecv error: recv got EOF from 127.0.0.1:40698 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260812 07:15:16.647997 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.641416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56322 (local address 127.21.181.125:43379)
0812 07:15:16.641596 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.641603 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.642134 (+ 531us) server_negotiation.cc:407] Connection header received
0812 07:15:16.642300 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.642305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.642376 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.642465 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.642917 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.643939 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.645913 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.646602 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.646731 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.647251 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.647393 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.647483 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.647669 (+ 186us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.647743 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 07:15:16.657898 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.655245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.21.181.124:39411)
0812 07:15:16.655511 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.655517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.656143 (+ 626us) server_negotiation.cc:407] Connection header received
0812 07:15:16.656454 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.656461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.656559 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.656643 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.657642 (+ 999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40702: BlockingRecv error: recv got EOF from 127.0.0.1:40702 (error 108)
Metrics: {"server-negotiator.queue_time_us":153}
I20260812 07:15:16.676970 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.669976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56338 (local address 127.21.181.125:43379)
0812 07:15:16.670168 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.670174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.670729 (+ 555us) server_negotiation.cc:407] Connection header received
0812 07:15:16.670932 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.670938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.671015 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.671113 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.671592 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.672749 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.674729 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.675404 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.675535 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.676175 (+ 640us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.676315 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.676416 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.676574 (+ 158us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.676659 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260812 07:15:16.686347 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.683680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40716 (local address 127.21.181.124:39411)
0812 07:15:16.683891 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.683897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.684538 (+ 641us) server_negotiation.cc:407] Connection header received
0812 07:15:16.684863 (+ 325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.684869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.684973 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.685053 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.686104 (+ 1051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40716: BlockingRecv error: recv got EOF from 127.0.0.1:40716 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:15:16.717718 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.710446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56342 (local address 127.21.181.125:43379)
0812 07:15:16.710663 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.710670 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.711248 (+ 578us) server_negotiation.cc:407] Connection header received
0812 07:15:16.711483 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.711489 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.711582 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.711724 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.712326 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.713530 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.715564 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.716278 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.716431 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.716945 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.717088 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.717203 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.717311 (+ 108us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.717417 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260812 07:15:16.728626 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.725976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.21.181.124:39411)
0812 07:15:16.726168 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.726175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.726722 (+ 547us) server_negotiation.cc:407] Connection header received
0812 07:15:16.727025 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.727032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.727128 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.727202 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.728368 (+ 1166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40718: BlockingRecv error: recv got EOF from 127.0.0.1:40718 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:15:16.770530 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.764562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.21.181.125:43379)
0812 07:15:16.764804 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.764810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.765095 (+ 285us) server_negotiation.cc:407] Connection header received
0812 07:15:16.765242 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.765248 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.765442 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.765572 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.766002 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.767224 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.768536 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.769206 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.769341 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.769755 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.769895 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.769991 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.770098 (+ 107us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.770177 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260812 07:15:16.778646 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.776887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40730 (local address 127.21.181.124:39411)
0812 07:15:16.777068 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.777074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.777402 (+ 328us) server_negotiation.cc:407] Connection header received
0812 07:15:16.777531 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.777536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.777638 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.777730 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.778423 (+ 693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40730: BlockingRecv error: recv got EOF from 127.0.0.1:40730 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260812 07:15:16.851174 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.846270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.21.181.125:43379)
0812 07:15:16.846475 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.846482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.846781 (+ 299us) server_negotiation.cc:407] Connection header received
0812 07:15:16.846985 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.846989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.847054 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.847132 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.847535 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.848298 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.849609 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.850011 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.850126 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.850518 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.850628 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.850707 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.850843 (+ 136us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.850946 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:15:16.857635 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.856008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40742 (local address 127.21.181.124:39411)
0812 07:15:16.856190 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.856196 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.856504 (+ 308us) server_negotiation.cc:407] Connection header received
0812 07:15:16.856660 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.856665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.856730 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.856810 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:16.857429 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40742: BlockingRecv error: recv got EOF from 127.0.0.1:40742 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260812 07:15:16.993836 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:16.988814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56366 (local address 127.21.181.125:43379)
0812 07:15:16.989048 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.989056 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.989361 (+ 305us) server_negotiation.cc:407] Connection header received
0812 07:15:16.989563 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.989567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.989636 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.989715 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:16.990191 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.990882 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.992226 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:16.992639 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:16.992753 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:16.993168 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:16.993275 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:16.993352 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:16.993508 (+ 156us) server_negotiation.cc:299] Negotiation successful
0812 07:15:16.993589 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:15:17.000460 31168 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:16.998726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40744 (local address 127.21.181.124:39411)
0812 07:15:16.998926 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:16.998931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:16.999186 (+ 255us) server_negotiation.cc:407] Connection header received
0812 07:15:16.999359 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:16.999363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:16.999443 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:16.999526 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:17.000266 (+ 740us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40744: BlockingRecv error: recv got EOF from 127.0.0.1:40744 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:15:17.019747 31121 catalog_manager.cc:1259] Loaded cluster ID: b7ab7a0f30c64de3af10b4c5837a46ab
I20260812 07:15:17.019827 31121 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: loading cluster ID for follower catalog manager: success
I20260812 07:15:17.022318 31121 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: acquiring CA information for follower catalog manager: success
I20260812 07:15:17.024017 31121 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d41e6bbd3d614a639a07bdb739a7c8e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:15:17.266893 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:17.261678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56372 (local address 127.21.181.125:43379)
0812 07:15:17.261853 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:17.261857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:17.262103 (+ 246us) server_negotiation.cc:407] Connection header received
0812 07:15:17.262270 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:17.262274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:17.262341 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:17.262425 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:17.262781 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.263623 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.265111 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.265544 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.265639 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:17.266168 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:17.266296 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:17.266405 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:17.266557 (+ 152us) server_negotiation.cc:299] Negotiation successful
0812 07:15:17.266639 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260812 07:15:17.281658 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:17.276421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40750 (local address 127.21.181.124:39411)
0812 07:15:17.276601 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:17.276605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:17.276778 (+ 173us) server_negotiation.cc:407] Connection header received
0812 07:15:17.276900 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:17.276903 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:17.277110 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:17.277205 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:17.277578 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.278312 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.279601 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.280055 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.280271 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:17.280692 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:17.281224 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:17.281341 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:17.281383 (+ 42us) server_negotiation.cc:299] Negotiation successful
0812 07:15:17.281449 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260812 07:15:17.294723 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:17.287173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.21.181.125:43379)
0812 07:15:17.287358 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:17.287364 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:17.288884 (+ 1520us) server_negotiation.cc:407] Connection header received
0812 07:15:17.289183 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:17.289189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:17.289276 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:17.289338 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:17.290240 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.290930 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.292915 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.293389 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.293503 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:17.293941 (+ 438us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:17.294067 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:17.294155 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:17.294346 (+ 191us) server_negotiation.cc:299] Negotiation successful
0812 07:15:17.294428 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260812 07:15:17.295437 31168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:17.287961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40758 (local address 127.21.181.124:39411)
0812 07:15:17.288140 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:17.288145 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:17.289330 (+ 1185us) server_negotiation.cc:407] Connection header received
0812 07:15:17.289483 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:17.289488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:17.289566 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:17.289652 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:17.290390 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.291374 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.293056 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.293613 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.293730 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:17.294207 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:17.294319 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:17.294400 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:17.295086 (+ 686us) server_negotiation.cc:299] Negotiation successful
0812 07:15:17.295164 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260812 07:15:17.296164 31433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:17.287112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39606 (local address 127.21.181.126:39275)
0812 07:15:17.288949 (+ 1837us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:17.288954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:17.288971 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:15:17.289083 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:17.289089 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:17.289297 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:17.289518 (+ 221us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:17.290192 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.291245 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.293371 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:17.293907 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:17.294024 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:17.294581 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:17.295015 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:17.295119 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:17.295765 (+ 646us) server_negotiation.cc:299] Negotiation successful
0812 07:15:17.295847 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":1625,"threads_started":1}
I20260812 07:15:17.300272 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30699
I20260812 07:15:17.301227 30807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:17.408739 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30699
I20260812 07:15:17.433923 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30833
I20260812 07:15:17.434931 30949 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:17.538772 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30833
I20260812 07:15:17.568123 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 30968
I20260812 07:15:17.569217 31084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:17.671814 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 30968
2026-08-12T07:15:17Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4773 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 12 07:15:17 dist-test-slave-lgr5 krb5kdc[31452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 12 07:15:17 dist-test-slave-lgr5 krb5kdc[31452](info): set up 2 sockets
Aug 12 07:15:17 dist-test-slave-lgr5 krb5kdc[31452](info): commencing operation
krb5kdc: starting...
W20260812 07:15:19.330120 22229 mini_kdc.cc:121] Time spent starting KDC: real 1.605s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/test-user.keytab.
Aug 12 07:15:19 dist-test-slave-lgr5 krb5kdc[31452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-12T07:15:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-12T07:15:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.21.181.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
Entry for principal kudu/127.21.181.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab.
I20260812 07:15:19.746644 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:46641
--webserver_interface=127.21.181.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:46641,127.21.181.125:33899,127.21.181.124:44941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260812 07:15:20.079025 31474 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:15:20.079324 31474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:20.079397 31474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:20.079476 31474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:20.086563 31474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:20.086645 31474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:20.086707 31474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:20.086756 31474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:20.086815 31474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:20.096262 31474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:46641,127.21.181.125:33899,127.21.181.124:44941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.126.keytab
--principal=kudu/127.21.181.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.126:46641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.181.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:20.097460 31474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:20.098709 31474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:20.110080 31487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:20.110018 31488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:20.110057 31492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:20.110826 31474 server_base.cc:1034] running on GCE node
I20260812 07:15:20.111491 31474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:20.112804 31474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:20.114037 31474 hybrid_clock.cc:648] HybridClock initialized: now 1786518920113958 us; error 38 us; skew 500 ppm
I20260812 07:15:20.114337 31474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:20 dist-test-slave-lgr5 krb5kdc[31452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:20.117267 31474 init.cc:377] Logged in from keytab as kudu/127.21.181.126@KRBTEST.COM (short username kudu)
I20260812 07:15:20.118621 31474 webserver.cc:460] Webserver started at http://127.21.181.126:43943/ using document root <none> and password file <none>
I20260812 07:15:20.119272 31474 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:20.119370 31474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:20.119673 31474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:20.122519 31474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data/instance:
uuid: "ded2c6d04eaf48529194e5405300246d"
format_stamp: "Formatted at 2026-08-12 07:15:20 on dist-test-slave-lgr5"
server_key: "d995ebfccf1ddc53140ccd061f71dbc4"
server_key_iv: "adc111cba4959cf1354abedf1ce30128"
server_key_version: "encryptionkey@0"
I20260812 07:15:20.123274 31474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal/instance:
uuid: "ded2c6d04eaf48529194e5405300246d"
format_stamp: "Formatted at 2026-08-12 07:15:20 on dist-test-slave-lgr5"
server_key: "d995ebfccf1ddc53140ccd061f71dbc4"
server_key_iv: "adc111cba4959cf1354abedf1ce30128"
server_key_version: "encryptionkey@0"
I20260812 07:15:20.128139 31474 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260812 07:15:20.131289 31503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:20.132506 31474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260812 07:15:20.132663 31474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal
uuid: "ded2c6d04eaf48529194e5405300246d"
format_stamp: "Formatted at 2026-08-12 07:15:20 on dist-test-slave-lgr5"
server_key: "d995ebfccf1ddc53140ccd061f71dbc4"
server_key_iv: "adc111cba4959cf1354abedf1ce30128"
server_key_version: "encryptionkey@0"
I20260812 07:15:20.132853 31474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:20.154551 31474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:20.158082 31474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:20.158314 31474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:20.175175 31474 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.126:46641
I20260812 07:15:20.175182 31607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.126:46641 every 8 connection(s)
I20260812 07:15:20.176409 31474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/data/info.pb
I20260812 07:15:20.178241 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31474
I20260812 07:15:20.178444 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-0/wal/instance
I20260812 07:15:20.179008 22229 external_mini_cluster.cc:1428] Setting key f3bfc1d6e537f6793e26e72c355bf1ee
I20260812 07:15:20.180404 31608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:20.188513 31608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:20 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:20.210937 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.191869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:46641 (local address 127.0.0.1:40026)
0812 07:15:20.192716 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.192767 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:20.194834 (+ 2067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:20.195762 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:20.195767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:20.196337 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:20.196674 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.196691 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.198764 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.198768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.199786 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.199798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.200996 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.201001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.201125 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.201690 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:20.201712 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:20.203741 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:20.207342 (+ 3601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.207352 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.207359 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.207725 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.208186 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.208189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.208192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.208339 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.209177 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:20.209182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:20.209355 (+ 173us) client_negotiation.cc:769] Sending connection context
0812 07:15:20.209556 (+ 201us) client_negotiation.cc:240] Negotiation successful
0812 07:15:20.209953 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":355,"threads_started":1}
I20260812 07:15:20.210978 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.191869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.21.181.126:46641)
0812 07:15:20.193541 (+ 1672us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:20.193547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:20.193593 (+ 46us) server_negotiation.cc:407] Connection header received
0812 07:15:20.195200 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:20.195208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:20.195546 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:20.195728 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:20.196885 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.198632 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.199925 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.200801 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.200975 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.203963 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:20.203973 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:20.203987 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:20.204025 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:20.207218 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.207849 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.207856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.207859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.208055 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.208444 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.208448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.208451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.209057 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:20.209232 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:20.209741 (+ 509us) server_negotiation.cc:299] Negotiation successful
0812 07:15:20.210133 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":1485,"spinlock_wait_cycles":25856,"thread_start_us":1261,"threads_started":1}
WARNING: no policy specified for kudu/127.21.181.125@KRBTEST.COM; defaulting to no policy
I20260812 07:15:20.214171 31608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:20.215399 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.214950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36808)
0812 07:15:20.215150 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.215267 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260812 07:15:20.215646 31507 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.125@KRBTEST.COM" created.
I20260812 07:15:20.217180 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.216772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36814)
0812 07:15:20.216942 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.217027 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:15:20.217422 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:33899: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab.
Entry for principal kudu/127.21.181.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab.
I20260812 07:15:20.250748 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:33899
--webserver_interface=127.21.181.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab
--principal=kudu/127.21.181.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:46641,127.21.181.125:33899,127.21.181.124:44941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:15:20.266738 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } attempt: 1
I20260812 07:15:20.267915 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.267459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36820)
0812 07:15:20.267666 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.267771 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260812 07:15:20.268965 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.268644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36832)
0812 07:15:20.268765 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.268835 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:15:20.269148 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:33899: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
I20260812 07:15:20.337361 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } attempt: 2
I20260812 07:15:20.338529 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.338066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36848)
0812 07:15:20.338288 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.338378 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:15:20.339514 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.339171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36864)
0812 07:15:20.339285 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.339367 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 07:15:20.339706 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:33899: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
I20260812 07:15:20.430936 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } attempt: 3
I20260812 07:15:20.432067 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.431654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36870)
0812 07:15:20.431839 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.431930 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260812 07:15:20.433054 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.432743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36872)
0812 07:15:20.432857 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.432922 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 07:15:20.433230 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:33899: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
W20260812 07:15:20.574142 31617 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:15:20.574450 31617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:20.574529 31617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:20.574613 31617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260812 07:15:20.576503 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } attempt: 4
I20260812 07:15:20.577715 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.577178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36882)
0812 07:15:20.577475 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.577577 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260812 07:15:20.578750 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.578414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36886)
0812 07:15:20.578545 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.578630 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260812 07:15:20.578933 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.125:33899: Network error: Client connection negotiation failed: client connection to 127.21.181.125:33899: connect: Connection refused (error 111)
W20260812 07:15:20.582190 31617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:20.582274 31617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:20.582351 31617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:20.582409 31617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:20.582477 31617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:20.591943 31617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:46641,127.21.181.125:33899,127.21.181.124:44941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.125.keytab
--principal=kudu/127.21.181.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.125:33899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.181.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:20.593109 31617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:20.594360 31617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:20.605391 31633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:20.605458 31628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:20.605691 31631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:20.606081 31617 server_base.cc:1034] running on GCE node
I20260812 07:15:20.606768 31617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:20.607982 31617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:20.609198 31617 hybrid_clock.cc:648] HybridClock initialized: now 1786518920609129 us; error 41 us; skew 500 ppm
I20260812 07:15:20.609500 31617 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:20 dist-test-slave-lgr5 krb5kdc[31452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:20.612347 31617 init.cc:377] Logged in from keytab as kudu/127.21.181.125@KRBTEST.COM (short username kudu)
I20260812 07:15:20.613675 31617 webserver.cc:460] Webserver started at http://127.21.181.125:43179/ using document root <none> and password file <none>
I20260812 07:15:20.614296 31617 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:20.614395 31617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:20.614699 31617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:20.617492 31617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data/instance:
uuid: "046941bd2fa74a278548c38a46c74e0b"
format_stamp: "Formatted at 2026-08-12 07:15:20 on dist-test-slave-lgr5"
server_key: "d3ed67cb03bd74bb26379c4a45742b9d"
server_key_iv: "48dd1e7732eda0f8928242eb92dc602e"
server_key_version: "encryptionkey@0"
I20260812 07:15:20.618245 31617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal/instance:
uuid: "046941bd2fa74a278548c38a46c74e0b"
format_stamp: "Formatted at 2026-08-12 07:15:20 on dist-test-slave-lgr5"
server_key: "d3ed67cb03bd74bb26379c4a45742b9d"
server_key_iv: "48dd1e7732eda0f8928242eb92dc602e"
server_key_version: "encryptionkey@0"
I20260812 07:15:20.622939 31617 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260812 07:15:20.626067 31651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:20.627194 31617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260812 07:15:20.627352 31617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal
uuid: "046941bd2fa74a278548c38a46c74e0b"
format_stamp: "Formatted at 2026-08-12 07:15:20 on dist-test-slave-lgr5"
server_key: "d3ed67cb03bd74bb26379c4a45742b9d"
server_key_iv: "48dd1e7732eda0f8928242eb92dc602e"
server_key_version: "encryptionkey@0"
I20260812 07:15:20.627540 31617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:20.640790 31617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:20.644160 31617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:20.644384 31617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:20.660831 31617 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.125:33899
I20260812 07:15:20.660862 31738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.125:33899 every 8 connection(s)
I20260812 07:15:20.661979 31617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/data/info.pb
I20260812 07:15:20.665284 31739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:20.666350 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31617
I20260812 07:15:20.666541 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-1/wal/instance
I20260812 07:15:20.667127 22229 external_mini_cluster.cc:1428] Setting key f9c74de129975e910c1db6606f5e01b7
I20260812 07:15:20.673171 31739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:20 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:20.692771 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.676057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.21.181.126:46641)
0812 07:15:20.676252 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:20.676257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:20.677278 (+ 1021us) server_negotiation.cc:407] Connection header received
0812 07:15:20.679621 (+ 2343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:20.679627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:20.679706 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:20.679798 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:20.681538 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.682373 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.683955 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.684445 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.684573 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.687612 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:20.687617 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:20.687621 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:20.687651 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:20.690055 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.690709 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.690715 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.690719 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.690832 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.691203 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.691208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.691211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.691468 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:20.691603 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:20.692250 (+ 647us) server_negotiation.cc:299] Negotiation successful
0812 07:15:20.692465 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260812 07:15:20.693583 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.676397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:46641 (local address 127.0.0.1:40028)
0812 07:15:20.677112 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.677166 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:20.679318 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:20.680034 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:20.680044 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:20.680720 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:20.681385 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.681406 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.682530 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.682535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.683806 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.683818 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.684665 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.684673 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.684882 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.685440 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:20.685467 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:20.687399 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:20.690184 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.690192 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.690206 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.690597 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.690933 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.690938 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.690942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.691100 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.691592 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:20.691599 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:20.691894 (+ 295us) client_negotiation.cc:769] Sending connection context
0812 07:15:20.692140 (+ 246us) client_negotiation.cc:240] Negotiation successful
0812 07:15:20.692589 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":274,"threads_started":1}
I20260812 07:15:20.695118 31739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.21.181.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.21.181.124@KRBTEST.COM" created.
Aug 12 07:15:20 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:15:20.707834 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.695791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36902)
0812 07:15:20.695955 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.695972 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:20.696058 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:20.696807 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:20.696811 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:20.697204 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:20.697319 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.697327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.698632 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.698638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.699472 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.699483 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.700322 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.700328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.700503 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.700974 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:20.700990 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:20.702788 (+ 1798us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:20.705671 (+ 2883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.705676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.705680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.706011 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.706335 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.706340 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.706344 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.706473 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.707005 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:20.707009 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:20.707132 (+ 123us) client_negotiation.cc:769] Sending connection context
0812 07:15:20.707242 (+ 110us) client_negotiation.cc:240] Negotiation successful
0812 07:15:20.707477 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260812 07:15:20.708068 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.695894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36902 (local address 127.21.181.125:33899)
0812 07:15:20.696447 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:20.696452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:20.696475 (+ 23us) server_negotiation.cc:407] Connection header received
0812 07:15:20.696521 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:20.696526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:20.696675 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:20.696793 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:20.697471 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.698513 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.699594 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.700189 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.700357 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.702969 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:20.702980 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:20.702991 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:20.703024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:20.705551 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.706124 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.706130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.706132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.706240 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.706568 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.706574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.706577 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.706876 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:20.707050 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:20.707397 (+ 347us) server_negotiation.cc:299] Negotiation successful
0812 07:15:20.707604 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":418,"thread_start_us":296,"threads_started":1}
I20260812 07:15:20.710191 31739 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:20.711236 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.710863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52428)
0812 07:15:20.711004 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.711101 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:15:20.711512 31657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:20.713001 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.712637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52436)
0812 07:15:20.712798 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.712887 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260812 07:15:20.713204 31739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.21.181.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab.
Entry for principal kudu/127.21.181.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab.
I20260812 07:15:20.732932 22229 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
/tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:44941
--webserver_interface=127.21.181.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab
--principal=kudu/127.21.181.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.21.181.84:37727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.181.126:46641,127.21.181.125:33899,127.21.181.124:44941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260812 07:15:20.762481 31739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 1
I20260812 07:15:20.763574 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.763149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52450)
0812 07:15:20.763325 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.763424 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:15:20.764519 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.764198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52452)
0812 07:15:20.764336 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.764410 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260812 07:15:20.764706 31739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:20.832877 31739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 2
I20260812 07:15:20.833945 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.833536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52458)
0812 07:15:20.833710 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.833799 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260812 07:15:20.834919 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.834612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52464)
0812 07:15:20.834746 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.834812 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:15:20.835144 31739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:20.878157 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } attempt: 5
Aug 12 07:15:20 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:15:20.890725 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.878895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36912 (local address 127.21.181.125:33899)
0812 07:15:20.879062 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:20.879067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:20.879155 (+ 88us) server_negotiation.cc:407] Connection header received
0812 07:15:20.879338 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:20.879343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:20.879422 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:20.879524 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:20.880338 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.881043 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.882171 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.882619 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.882730 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.885861 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:20.885866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:20.885869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:20.885902 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:20.888348 (+ 2446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.889097 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.889100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.889102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.889214 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:20.889583 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:20.889586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:20.889587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:20.889835 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:20.889941 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:20.890352 (+ 411us) server_negotiation.cc:299] Negotiation successful
0812 07:15:20.890512 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:15:20.890956 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.878838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36912)
0812 07:15:20.879060 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.879082 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:20.879215 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:20.879564 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:20.879569 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:20.879964 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:20.880190 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.880200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.881173 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.881178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.882021 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:20.882034 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:20.882759 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:20.882764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:20.882962 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:20.883519 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:20.883538 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:20.885680 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:20.888489 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.888494 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.888498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.888998 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.889306 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:20.889311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:20.889315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:20.889495 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:20.889941 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:20.889946 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:20.890125 (+ 179us) client_negotiation.cc:769] Sending connection context
0812 07:15:20.890284 (+ 159us) client_negotiation.cc:240] Negotiation successful
0812 07:15:20.890580 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260812 07:15:20.891937 31608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } has no permanent_uuid. Determining permanent_uuid...
I20260812 07:15:20.893004 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.892651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52470)
0812 07:15:20.892801 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.892873 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260812 07:15:20.893915 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.893632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52474)
0812 07:15:20.893738 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.893821 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260812 07:15:20.894127 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:20.926329 31739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 3
I20260812 07:15:20.927424 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.926995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52482)
0812 07:15:20.927174 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.927272 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:15:20.928579 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.927999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52496)
0812 07:15:20.928291 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.928399 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
W20260812 07:15:20.928828 31739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:20.945312 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 1
I20260812 07:15:20.946228 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.945847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52508)
0812 07:15:20.945999 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.946078 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260812 07:15:20.947114 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:20.946833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52518)
0812 07:15:20.946946 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:20.947013 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260812 07:15:20.947324 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:21.015518 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 2
I20260812 07:15:21.016686 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.016230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52530)
0812 07:15:21.016414 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.016526 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260812 07:15:21.017639 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.017341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52542)
0812 07:15:21.017466 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.017539 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260812 07:15:21.017856 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
W20260812 07:15:21.058342 31750 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 07:15:21.058622 31750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 07:15:21.058714 31750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 07:15:21.058815 31750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260812 07:15:21.065726 31750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 07:15:21.065801 31750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260812 07:15:21.065888 31750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260812 07:15:21.065948 31750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 07:15:21.066046 31750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260812 07:15:21.072026 31739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 4
I20260812 07:15:21.073138 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.072586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52558)
0812 07:15:21.072827 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.072931 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260812 07:15:21.074071 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.073745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52572)
0812 07:15:21.073875 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.073955 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260812 07:15:21.074290 31739 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:21.075738 31750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.181.84:37727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.181.126:46641,127.21.181.125:33899,127.21.181.124:44941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/krb5kdc/kudu_127.21.181.124.keytab
--principal=kudu/127.21.181.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.181.124:44941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.181.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 12 Aug 2026 06:03:33 UTC on ccycloud.quasar-yqmzqh.root.comops.site
build id 4581991
ASAN enabled
I20260812 07:15:21.076833 31750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 07:15:21.078068 31750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260812 07:15:21.089083 31755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:21.089047 31756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 07:15:21.088949 31762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 07:15:21.089840 31750 server_base.cc:1034] running on GCE node
I20260812 07:15:21.090584 31750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 07:15:21.091784 31750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 07:15:21.093005 31750 hybrid_clock.cc:648] HybridClock initialized: now 1786518921092936 us; error 24 us; skew 500 ppm
I20260812 07:15:21.093303 31750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518921, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260812 07:15:21.096289 31750 init.cc:377] Logged in from keytab as kudu/127.21.181.124@KRBTEST.COM (short username kudu)
I20260812 07:15:21.097570 31750 webserver.cc:460] Webserver started at http://127.21.181.124:38907/ using document root <none> and password file <none>
I20260812 07:15:21.098199 31750 fs_manager.cc:359] Metadata directory not provided
I20260812 07:15:21.098300 31750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 07:15:21.098609 31750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 07:15:21.101431 31750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data/instance:
uuid: "2d95fa1ee1ee4cbea48d9c200918338e"
format_stamp: "Formatted at 2026-08-12 07:15:21 on dist-test-slave-lgr5"
server_key: "bf61aeb3ad97d10f31e962b8ace9ccde"
server_key_iv: "09983204af170f0384ca715f9d1c738b"
server_key_version: "encryptionkey@0"
I20260812 07:15:21.102195 31750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal/instance:
uuid: "2d95fa1ee1ee4cbea48d9c200918338e"
format_stamp: "Formatted at 2026-08-12 07:15:21 on dist-test-slave-lgr5"
server_key: "bf61aeb3ad97d10f31e962b8ace9ccde"
server_key_iv: "09983204af170f0384ca715f9d1c738b"
server_key_version: "encryptionkey@0"
I20260812 07:15:21.106973 31750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260812 07:15:21.110280 31778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260812 07:15:21.111472 31750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260812 07:15:21.111635 31750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data,/tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal
uuid: "2d95fa1ee1ee4cbea48d9c200918338e"
format_stamp: "Formatted at 2026-08-12 07:15:21 on dist-test-slave-lgr5"
server_key: "bf61aeb3ad97d10f31e962b8ace9ccde"
server_key_iv: "09983204af170f0384ca715f9d1c738b"
server_key_version: "encryptionkey@0"
I20260812 07:15:21.111819 31750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 07:15:21.122951 31750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260812 07:15:21.124033 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 3
I20260812 07:15:21.125113 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.124634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52586)
0812 07:15:21.124845 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.124926 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260812 07:15:21.126188 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.125749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52602)
0812 07:15:21.125953 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.126043 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260812 07:15:21.126436 31608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.21.181.124:44941: Network error: Client connection negotiation failed: client connection to 127.21.181.124:44941: connect: Connection refused (error 111)
I20260812 07:15:21.127177 31750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 07:15:21.127408 31750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 07:15:21.144023 31750 rpc_server.cc:307] RPC server started. Bound to: 127.21.181.124:44941
I20260812 07:15:21.144042 31873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.21.181.124:44941 every 8 connection(s)
I20260812 07:15:21.145272 31750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/data/info.pb
I20260812 07:15:21.148840 31874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 07:15:21.148826 22229 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu as pid 31750
I20260812 07:15:21.149024 22229 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCYQ_m9/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786518562800324-22229-0/minicluster-data/master-2/wal/instance
I20260812 07:15:21.149560 22229 external_mini_cluster.cc:1428] Setting key 954b849987bdfb251bc3489286c3e6f4
I20260812 07:15:21.158046 31874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518921, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
I20260812 07:15:21.171234 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.151225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52608 (local address 127.21.181.124:44941)
0812 07:15:21.152009 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.152020 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.152058 (+ 38us) server_negotiation.cc:407] Connection header received
0812 07:15:21.153710 (+ 1652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.153734 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.154103 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.154413 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.156112 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.157342 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.159102 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.159731 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.159850 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.164269 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.164277 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.164289 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.164321 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.167260 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.167984 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.167991 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.167996 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.168181 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.168551 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.168556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.168560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.169163 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.169374 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.169855 (+ 481us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.170213 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":366,"threads_started":1}
I20260812 07:15:21.174494 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.160705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.126:46641 (local address 127.0.0.1:40042)
0812 07:15:21.161402 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.161450 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:21.161591 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:21.161941 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:21.161946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:21.162476 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:21.162704 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.162712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.163990 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.163993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.165086 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.165098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.165838 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.165849 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.166012 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.166541 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:21.166557 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:21.168848 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:21.171907 (+ 3059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.171916 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.171922 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.172443 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.172811 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.172816 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.172818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.172938 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.173501 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:21.173509 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:21.173658 (+ 149us) client_negotiation.cc:769] Sending connection context
0812 07:15:21.173804 (+ 146us) client_negotiation.cc:240] Negotiation successful
0812 07:15:21.174097 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":343,"threads_started":1}
I20260812 07:15:21.174539 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.160661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40042 (local address 127.21.181.126:46641)
0812 07:15:21.160870 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.160876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.161547 (+ 671us) server_negotiation.cc:407] Connection header received
0812 07:15:21.161706 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.161711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.161794 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.161902 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.162850 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.163845 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.165229 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.165648 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.165757 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.169021 (+ 3264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.169026 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.169029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.169061 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.171793 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.172566 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.172571 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.172573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.172704 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.173050 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.173054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.173056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.173350 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.173498 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.173954 (+ 456us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.174179 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260812 07:15:21.180367 31874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.126@KRBTEST.COM
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518921, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
I20260812 07:15:21.188551 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.174076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40056 (local address 127.21.181.126:46641)
0812 07:15:21.174735 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.174740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.174758 (+ 18us) server_negotiation.cc:407] Connection header received
0812 07:15:21.174811 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.174815 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.174886 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.174995 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.176116 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.176964 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.178133 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.178569 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.178670 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.182927 (+ 4257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.182932 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.182935 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.182963 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.185395 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.186091 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.186095 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.186097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.186228 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.186591 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.186593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.186595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.186826 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.187033 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.187877 (+ 844us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.188074 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":4946,"threads_started":1}
I20260812 07:15:21.193487 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.181155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.125:33899 (local address 127.0.0.1:36916)
0812 07:15:21.181389 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.181405 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:21.181505 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:21.181887 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:21.181892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:21.182324 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:21.182511 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.182523 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.183475 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.183481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.184513 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.184526 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.185211 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.185218 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.185384 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.185826 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:21.185847 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:21.187921 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:21.191350 (+ 3429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.191353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.191356 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.191825 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.192201 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.192206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.192210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.192376 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.192859 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:21.192863 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:21.192990 (+ 127us) client_negotiation.cc:769] Sending connection context
0812 07:15:21.193104 (+ 114us) client_negotiation.cc:240] Negotiation successful
0812 07:15:21.193280 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260812 07:15:21.193807 31747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.181171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36916 (local address 127.21.181.125:33899)
0812 07:15:21.181352 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.181358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.181601 (+ 243us) server_negotiation.cc:407] Connection header received
0812 07:15:21.181663 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.181668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.181740 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.181828 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.182678 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.183343 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.184659 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.185071 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.185179 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.188172 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.188179 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.188184 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.188223 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.191238 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.191932 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.191935 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.191938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.192060 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.192472 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.192475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.192478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.192735 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.192885 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.193239 (+ 354us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.193450 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:15:21.194625 31874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } has no permanent_uuid. Determining permanent_uuid...
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.21.181.125@KRBTEST.COM
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518921, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.124@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:15:21.205895 31887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.190567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.21.181.125:33899)
0812 07:15:21.191166 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.191172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.191196 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:15:21.191242 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.191246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.191307 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.191396 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.192821 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.193921 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.195301 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.195922 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.196015 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.200609 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.200616 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.200621 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.200658 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.203327 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.204031 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.204035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.204038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.204191 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.204583 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.204585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.204588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.204824 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.205002 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.205323 (+ 321us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.205541 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":240,"threads_started":1}
I20260812 07:15:21.207579 31882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.195264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52624)
0812 07:15:21.195463 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.195476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:21.195553 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:21.195797 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:21.195800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:21.196221 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:21.196378 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.196389 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.197360 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.197366 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.198684 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.198700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.199452 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.199456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.199620 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.200133 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:21.200154 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:21.202275 (+ 2121us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:21.205185 (+ 2910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.205189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.205191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.205633 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.205948 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.205952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.205956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.206120 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.206691 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:21.206694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:21.206826 (+ 132us) client_negotiation.cc:769] Sending connection context
0812 07:15:21.206963 (+ 137us) client_negotiation.cc:240] Negotiation successful
0812 07:15:21.207251 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260812 07:15:21.207787 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.195265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.21.181.124:44941)
0812 07:15:21.195426 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.195431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.195528 (+ 97us) server_negotiation.cc:407] Connection header received
0812 07:15:21.195624 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.195629 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.195696 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.195789 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.196515 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.197231 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.198809 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.199294 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.199425 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.202443 (+ 3018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.202447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.202450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.202479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.205059 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.205736 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.205740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.205742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.205849 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.206224 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.206228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.206230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.206558 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.206699 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.207085 (+ 386us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.207423 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":63}
I20260812 07:15:21.210739 22229 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260812 07:15:21.212770 31874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: Bootstrap starting.
I20260812 07:15:21.212962 31557 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ded2c6d04eaf48529194e5405300246d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40056
I20260812 07:15:21.216053 31874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:21.217409 31874 log.cc:824] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:21.220391 31874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: No bootstrap required, opened a new log
I20260812 07:15:21.230904 31874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.231298 31874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d95fa1ee1ee4cbea48d9c200918338e, State: Initialized, Role: FOLLOWER
I20260812 07:15:21.231823 31874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.232808 31889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.233109 31889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.233803 31874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: configured and running, proceeding with master startup.
W20260812 07:15:21.242588 31911 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 07:15:21.242709 31911 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 07:15:21.240372 31750 master_runner.cc:186] Error getting master registration for 127.21.181.126:46641: OK, instance_id { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" instance_seqno: 1786518920170312 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:15:21.251154 31750 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:15:21.303661 31608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 4
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.126@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:15:21.317147 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.304545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.21.181.124:44941)
0812 07:15:21.304758 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.304765 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.304789 (+ 24us) server_negotiation.cc:407] Connection header received
0812 07:15:21.304938 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.304945 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.305014 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.305100 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.305895 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.306799 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.308044 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.308580 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.308691 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.311869 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.311875 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.311878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.311911 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.314490 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.315277 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.315281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.315283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.315424 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.315816 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.315821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.315824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.316197 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.316325 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.316725 (+ 400us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.316883 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260812 07:15:21.317297 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.304463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52640)
0812 07:15:21.304673 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.304692 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:21.304798 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:21.305164 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:21.305168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:21.305573 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:21.305745 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.305755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.306943 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.306949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.307899 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.307912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.308723 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.308729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.308929 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.309473 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:21.309494 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:21.311683 (+ 2189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:21.314605 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.314611 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.314615 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.315172 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.315521 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.315526 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.315530 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.315697 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.316299 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:21.316304 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:21.316481 (+ 177us) client_negotiation.cc:769] Sending connection context
0812 07:15:21.316638 (+ 157us) client_negotiation.cc:240] Negotiation successful
0812 07:15:21.316916 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260812 07:15:21.321862 31608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d: Bootstrap starting.
I20260812 07:15:21.325078 31608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:21.326092 31608 log.cc:824] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:21.328846 31608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d: No bootstrap required, opened a new log
I20260812 07:15:21.333937 31608 raft_consensus.cc:348] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.334283 31608 raft_consensus.cc:729] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ded2c6d04eaf48529194e5405300246d, State: Initialized, Role: FOLLOWER
I20260812 07:15:21.334728 31608 consensus_queue.cc:260] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.335635 31914 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.336001 31914 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.336810 31608 sys_catalog.cc:564] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:21.337013 31557 raft_consensus.cc:482] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 07:15:21.337246 31557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:21.340744 31557 raft_consensus.cc:504] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.343161 31557 leader_election.cc:288] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [CANDIDATE]: Term 1 election: Requested vote from peers 046941bd2fa74a278548c38a46c74e0b (127.21.181.125:33899), 2d95fa1ee1ee4cbea48d9c200918338e (127.21.181.124:44941)
I20260812 07:15:21.343171 31828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ded2c6d04eaf48529194e5405300246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2d95fa1ee1ee4cbea48d9c200918338e"
I20260812 07:15:21.343638 31828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:21.347332 31828 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ded2c6d04eaf48529194e5405300246d in term 1.
I20260812 07:15:21.348042 31513 leader_election.cc:302] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d95fa1ee1ee4cbea48d9c200918338e, ded2c6d04eaf48529194e5405300246d; no voters:
I20260812 07:15:21.348536 31914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 07:15:21.349303 31914 raft_consensus.cc:686] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [term 1 LEADER]: Becoming Leader. State: Replica: ded2c6d04eaf48529194e5405300246d, State: Running, Role: LEADER
I20260812 07:15:21.349932 31914 consensus_queue.cc:237] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.362761 31928 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded2c6d04eaf48529194e5405300246d. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.363160 31928 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:21.364030 31937 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260812 07:15:21.373473 31739 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } attempt: 5
W20260812 07:15:21.379292 31791 tablet.cc:2369] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 12 07:15:21 dist-test-slave-lgr5 krb5kdc[31452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786518920, etypes {rep=17 tkt=17 ses=17}, kudu/127.21.181.125@KRBTEST.COM for kudu/127.21.181.124@KRBTEST.COM
I20260812 07:15:21.392870 31887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.363619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36942 (local address 127.21.181.125:33899)
0812 07:15:21.363811 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.363817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.363836 (+ 19us) server_negotiation.cc:407] Connection header received
0812 07:15:21.363897 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.363902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.363976 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.364124 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.367754 (+ 3630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.368872 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.371655 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.372331 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.372425 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.382775 (+ 10350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.382782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.382787 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.382823 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.386367 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.389394 (+ 3027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.389400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.389403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.389558 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.390912 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.390917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.390921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.391293 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.391549 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.392292 (+ 743us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.392515 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260812 07:15:21.406277 31742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.374036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.21.181.124:44941 (local address 127.0.0.1:52658)
0812 07:15:21.374336 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0812 07:15:21.374353 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0812 07:15:21.374444 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0812 07:15:21.376441 (+ 1997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0812 07:15:21.376447 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0812 07:15:21.376981 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0812 07:15:21.377173 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.377184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.383712 (+ 6528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.383730 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.385331 (+ 1601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0812 07:15:21.385346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.389115 (+ 3769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.389122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0812 07:15:21.389374 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.390094 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0812 07:15:21.390116 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0812 07:15:21.392931 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0812 07:15:21.402263 (+ 9332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.402269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.402274 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.402764 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.403679 (+ 915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0812 07:15:21.403685 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0812 07:15:21.403689 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0812 07:15:21.403883 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0812 07:15:21.405267 (+ 1384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0812 07:15:21.405273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0812 07:15:21.405449 (+ 176us) client_negotiation.cc:769] Sending connection context
0812 07:15:21.405583 (+ 134us) client_negotiation.cc:240] Negotiation successful
0812 07:15:21.405876 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"spinlock_wait_cycles":24960}
I20260812 07:15:21.408938 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.374762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.21.181.124:44941)
0812 07:15:21.375817 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.375852 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.375879 (+ 27us) server_negotiation.cc:407] Connection header received
0812 07:15:21.375998 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.376004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.376182 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.377394 (+ 1212us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.377909 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.383064 (+ 5155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.385614 (+ 2550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.388733 (+ 3119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.391803 (+ 3070us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.393389 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.393434 (+ 45us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.393442 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.393527 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.402019 (+ 8492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.403031 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.403038 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.403042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.403456 (+ 414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.404130 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.404136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.404140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.405029 (+ 889us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.407081 (+ 2052us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.407390 (+ 309us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.407944 (+ 554us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":839,"thread_start_us":475,"threads_started":1}
I20260812 07:15:21.411211 31937 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260812 07:15:21.414446 31515 tablet.cc:2369] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 07:15:21.417585 31739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: Bootstrap starting.
I20260812 07:15:21.423444 31739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: Neither blocks nor log segments found. Creating new log.
I20260812 07:15:21.424767 31880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.364727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52646 (local address 127.21.181.124:44941)
0812 07:15:21.364923 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.364930 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.381978 (+ 17048us) server_negotiation.cc:407] Connection header received
0812 07:15:21.382419 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.382425 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.382504 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.382573 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.393870 (+ 11297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.394906 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.399012 (+ 4106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.399724 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.399814 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.412213 (+ 12399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.412220 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.412225 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.412267 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.415928 (+ 3661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.416901 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.416907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.416911 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.417080 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.418599 (+ 1519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.418605 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.418608 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.418966 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.419215 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.424175 (+ 4960us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.424413 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260812 07:15:21.425587 31739 log.cc:824] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: Log is configured to *not* fsync() on all Append() calls
I20260812 07:15:21.428961 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:21.365433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40068 (local address 127.21.181.126:46641)
0812 07:15:21.365593 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.365599 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.390646 (+ 25047us) server_negotiation.cc:407] Connection header received
0812 07:15:21.395105 (+ 4459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.395110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.395189 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.395259 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.400740 (+ 5481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.401858 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.406142 (+ 4284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:21.406888 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:21.406984 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:21.419742 (+ 12758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0812 07:15:21.419749 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0812 07:15:21.419753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0812 07:15:21.419794 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0812 07:15:21.423583 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.426415 (+ 2832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.426421 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.426424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.426638 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0812 07:15:21.427128 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0812 07:15:21.427133 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0812 07:15:21.427136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0812 07:15:21.427517 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0812 07:15:21.427772 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:21.428342 (+ 570us) server_negotiation.cc:299] Negotiation successful
0812 07:15:21.428582 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260812 07:15:21.420034 31474 master_runner.cc:186] Error getting master registration for 127.21.181.125:33899: OK, instance_id { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" instance_seqno: 1786518920656242 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 07:15:21.431550 31474 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260812 07:15:21.450995 31739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: No bootstrap required, opened a new log
I20260812 07:15:21.457770 31828 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [term 1 FOLLOWER]: Refusing update from remote peer ded2c6d04eaf48529194e5405300246d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:15:21.458792 31914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:15:21.461424 31739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.461810 31739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046941bd2fa74a278548c38a46c74e0b, State: Initialized, Role: FOLLOWER
I20260812 07:15:21.462277 31739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } }
I20260812 07:15:21.469169 31955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.469590 31955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.471271 31739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: configured and running, proceeding with master startup.
I20260812 07:15:21.471321 31702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ded2c6d04eaf48529194e5405300246d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "046941bd2fa74a278548c38a46c74e0b"
I20260812 07:15:21.471638 31701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [term 0 FOLLOWER]: Advancing to term 1
I20260812 07:15:21.475548 31701 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [term 1 FOLLOWER]: Refusing update from remote peer ded2c6d04eaf48529194e5405300246d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 07:15:21.476658 31914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [LEADER]: Connected to new peer: Peer: permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 07:15:21.479559 31702 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ded2c6d04eaf48529194e5405300246d in term 1.
I20260812 07:15:21.501904 31955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded2c6d04eaf48529194e5405300246d. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.502207 31955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.507715 31914 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.508148 31914 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:21.510799 31889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ded2c6d04eaf48529194e5405300246d. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.511127 31889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.511608 31937 catalog_manager.cc:1347] Generated new cluster ID: c66851e9e74541228a7d5a8cd1457aad
I20260812 07:15:21.511703 31937 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260812 07:15:21.513752 31928 sys_catalog.cc:455] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.514014 31928 sys_catalog.cc:458] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d [sys.catalog]: This master's current role is: LEADER
I20260812 07:15:21.520393 31956 mvcc.cc:205] Tried to move back new op lower bound from 7317581502248493056 to 7317581501854736384. Current Snapshot: MvccSnapshot[applied={T|T < 7317581502248493056}]
I20260812 07:15:21.523909 31955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.524127 31955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.528072 31889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ded2c6d04eaf48529194e5405300246d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ded2c6d04eaf48529194e5405300246d" member_type: VOTER last_known_addr { host: "127.21.181.126" port: 46641 } } peers { permanent_uuid: "046941bd2fa74a278548c38a46c74e0b" member_type: VOTER last_known_addr { host: "127.21.181.125" port: 33899 } } peers { permanent_uuid: "2d95fa1ee1ee4cbea48d9c200918338e" member_type: VOTER last_known_addr { host: "127.21.181.124" port: 44941 } } }
I20260812 07:15:21.529166 31889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e [sys.catalog]: This master's current role is: FOLLOWER
I20260812 07:15:21.547032 31937 catalog_manager.cc:1370] Generated new certificate authority record
I20260812 07:15:21.547509 31987 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260812 07:15:21.548797 31937 catalog_manager.cc:1504] Loading token signing keys...
I20260812 07:15:21.565534 31937 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ded2c6d04eaf48529194e5405300246d: Generated new TSK 0
I20260812 07:15:21.568897 31937 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260812 07:15:21.656901 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.643627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36954 (local address 127.21.181.125:33899)
0812 07:15:21.643791 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.643796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.654333 (+ 10537us) server_negotiation.cc:407] Connection header received
0812 07:15:21.655329 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.655334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.655414 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.655487 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.656632 (+ 1145us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36954: BlockingRecv error: recv got EOF from 127.0.0.1:36954 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260812 07:15:21.656999 31880 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.651173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.21.181.124:44941)
0812 07:15:21.651366 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.651373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.655294 (+ 3921us) server_negotiation.cc:407] Connection header received
0812 07:15:21.655678 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.655683 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.655759 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.655840 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.656758 (+ 918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52672: BlockingRecv error: recv got EOF from 127.0.0.1:52672 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260812 07:15:21.658553 31612 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.643483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40074 (local address 127.21.181.126:46641)
0812 07:15:21.643714 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.643720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.654762 (+ 11042us) server_negotiation.cc:407] Connection header received
0812 07:15:21.654907 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.654913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.655002 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.655118 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.658304 (+ 3186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40074: BlockingRecv error: recv got EOF from 127.0.0.1:40074 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260812 07:15:21.694375 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.692431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36958 (local address 127.21.181.125:33899)
0812 07:15:21.692682 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.692688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.692993 (+ 305us) server_negotiation.cc:407] Connection header received
0812 07:15:21.693218 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.693222 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.693330 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.693418 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.694169 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36958: BlockingRecv error: recv got EOF from 127.0.0.1:36958 (error 108)
Metrics: {"server-negotiator.queue_time_us":118}
W20260812 07:15:21.700702 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.698735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.21.181.125:33899)
0812 07:15:21.698940 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.698947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.699388 (+ 441us) server_negotiation.cc:407] Connection header received
0812 07:15:21.699566 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.699572 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.699639 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.699718 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.700484 (+ 766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36974: BlockingRecv error: recv got EOF from 127.0.0.1:36974 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260812 07:15:21.708261 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.706211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36978 (local address 127.21.181.125:33899)
0812 07:15:21.706364 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.706370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.706783 (+ 413us) server_negotiation.cc:407] Connection header received
0812 07:15:21.707233 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.707237 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.707297 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.707387 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.708058 (+ 671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36978: BlockingRecv error: recv got EOF from 127.0.0.1:36978 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260812 07:15:21.719600 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.717565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.21.181.125:33899)
0812 07:15:21.717761 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.717767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.718242 (+ 475us) server_negotiation.cc:407] Connection header received
0812 07:15:21.718399 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.718404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.718470 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.718557 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.719368 (+ 811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36982: BlockingRecv error: recv got EOF from 127.0.0.1:36982 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260812 07:15:21.732122 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.730074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36988 (local address 127.21.181.125:33899)
0812 07:15:21.730294 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.730300 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.730819 (+ 519us) server_negotiation.cc:407] Connection header received
0812 07:15:21.730993 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.730999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.731061 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.731136 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.731898 (+ 762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36988: BlockingRecv error: recv got EOF from 127.0.0.1:36988 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260812 07:15:21.752748 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.750864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37002 (local address 127.21.181.125:33899)
0812 07:15:21.751062 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.751069 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.751501 (+ 432us) server_negotiation.cc:407] Connection header received
0812 07:15:21.751669 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.751672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.751733 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.751810 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.752536 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37002: BlockingRecv error: recv got EOF from 127.0.0.1:37002 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260812 07:15:21.789565 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.787806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37014 (local address 127.21.181.125:33899)
0812 07:15:21.788000 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.788006 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.788342 (+ 336us) server_negotiation.cc:407] Connection header received
0812 07:15:21.788490 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.788495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.788587 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.788707 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.789345 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37014: BlockingRecv error: recv got EOF from 127.0.0.1:37014 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260812 07:15:21.858642 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.856757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37026 (local address 127.21.181.125:33899)
0812 07:15:21.856982 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.856989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.857297 (+ 308us) server_negotiation.cc:407] Connection header received
0812 07:15:21.857469 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.857475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.857578 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.857713 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.858405 (+ 692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37026: BlockingRecv error: recv got EOF from 127.0.0.1:37026 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260812 07:15:21.995296 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:21.993602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37028 (local address 127.21.181.125:33899)
0812 07:15:21.993760 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:21.993765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:21.994070 (+ 305us) server_negotiation.cc:407] Connection header received
0812 07:15:21.994259 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:21.994262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:21.994350 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:21.994431 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:21.995072 (+ 641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37028: BlockingRecv error: recv got EOF from 127.0.0.1:37028 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260812 07:15:22.244736 31911 catalog_manager.cc:1259] Loaded cluster ID: c66851e9e74541228a7d5a8cd1457aad
I20260812 07:15:22.244834 31911 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: loading cluster ID for follower catalog manager: success
I20260812 07:15:22.247227 31911 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: acquiring CA information for follower catalog manager: success
I20260812 07:15:22.248948 31911 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d95fa1ee1ee4cbea48d9c200918338e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260812 07:15:22.256525 31887 negotiation.cc:337] Failed RPC negotiation. Trace:
0812 07:15:22.254771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37040 (local address 127.21.181.125:33899)
0812 07:15:22.254987 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:22.255006 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:22.255206 (+ 200us) server_negotiation.cc:407] Connection header received
0812 07:15:22.255409 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:22.255413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:22.255485 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:22.255575 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0812 07:15:22.256298 (+ 723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37040: BlockingRecv error: recv got EOF from 127.0.0.1:37040 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260812 07:15:22.551517 31987 catalog_manager.cc:1259] Loaded cluster ID: c66851e9e74541228a7d5a8cd1457aad
I20260812 07:15:22.551640 31987 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: loading cluster ID for follower catalog manager: success
I20260812 07:15:22.554217 31987 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: acquiring CA information for follower catalog manager: success
I20260812 07:15:22.556022 31987 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 046941bd2fa74a278548c38a46c74e0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 07:15:22.780679 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:22.773424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.21.181.125:33899)
0812 07:15:22.773986 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:22.774002 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:22.774249 (+ 247us) server_negotiation.cc:407] Connection header received
0812 07:15:22.774418 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:22.774424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:22.774630 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:22.774794 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:22.775192 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.776481 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.777875 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.778628 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.778796 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:22.779262 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:22.779975 (+ 713us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:22.780162 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:22.780217 (+ 55us) server_negotiation.cc:299] Negotiation successful
0812 07:15:22.780300 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":262,"threads_started":1}
I20260812 07:15:22.793036 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:22.786937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.21.181.124:44941)
0812 07:15:22.787522 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:22.787527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:22.787542 (+ 15us) server_negotiation.cc:407] Connection header received
0812 07:15:22.787631 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:22.787634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:22.787762 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:22.787890 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:22.788407 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.789076 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.790901 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.791289 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.791399 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:22.791954 (+ 555us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:22.792439 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:22.792539 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:22.792682 (+ 143us) server_negotiation.cc:299] Negotiation successful
0812 07:15:22.792745 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":387,"threads_started":1}
I20260812 07:15:22.807559 32100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:22.799521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.21.181.125:33899)
0812 07:15:22.799686 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:22.799692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:22.800930 (+ 1238us) server_negotiation.cc:407] Connection header received
0812 07:15:22.801208 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:22.801214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:22.801299 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:22.801361 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:22.802063 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.803135 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.805239 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.805946 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.806052 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:22.806763 (+ 711us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:22.806887 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:22.806946 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:22.807171 (+ 225us) server_negotiation.cc:299] Negotiation successful
0812 07:15:22.807253 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260812 07:15:22.809629 32121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:22.799554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.21.181.126:46641)
0812 07:15:22.802260 (+ 2706us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:22.802266 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:22.802283 (+ 17us) server_negotiation.cc:407] Connection header received
0812 07:15:22.802386 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:22.802390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:22.802532 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:22.802673 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:22.803321 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.804431 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.806685 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.807356 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.807974 (+ 618us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:22.808183 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:22.808668 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:22.809241 (+ 573us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:22.809289 (+ 48us) server_negotiation.cc:299] Negotiation successful
0812 07:15:22.809369 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2600,"thread_start_us":2502,"threads_started":1}
I20260812 07:15:22.813620 32109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0812 07:15:22.800853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.21.181.124:44941)
0812 07:15:22.802259 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0812 07:15:22.802264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0812 07:15:22.808380 (+ 6116us) server_negotiation.cc:407] Connection header received
0812 07:15:22.808623 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0812 07:15:22.808628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0812 07:15:22.808722 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0812 07:15:22.808787 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0812 07:15:22.809320 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.810297 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.811732 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0812 07:15:22.812380 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0812 07:15:22.812521 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0812 07:15:22.812950 (+ 429us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0812 07:15:22.813040 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0812 07:15:22.813152 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0812 07:15:22.813237 (+ 85us) server_negotiation.cc:299] Negotiation successful
0812 07:15:22.813310 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296}
I20260812 07:15:22.813839 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31474
I20260812 07:15:22.814791 31602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:22.943167 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31474
I20260812 07:15:22.971485 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31617
I20260812 07:15:22.972611 31734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:23.073202 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31617
I20260812 07:15:23.099189 22229 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu pid 31750
I20260812 07:15:23.100251 31861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260812 07:15:23.205905 22229 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCYQ_m9/build/asan/bin/kudu with pid 31750
2026-08-12T07:15:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5526 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10300 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (360369 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST